2026-02-19T12:40:41.038 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-19T12:40:41.050 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-19T12:40:41.138 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/nmordech-2026-02-19_12:10:46-rados-wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle-distro-default-trial/58618 branch: wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} email: null first_in_suite: false flavor: default job_id: '58618' kernel: branch: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: trial name: nmordech-2026-02-19_12:10:46-rados-wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 2fc421c83f0b86ea21b9e4289cad0f7957e09eb1 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: distro-suffix: rockylinux-10 install: ceph: flavor: default sha1: 2fc421c83f0b86ea21b9e4289cad0f7957e09eb1 workunit: branch: wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle-suite sha1: 624b9a7bcb2064ce17206482df6a1da007ff7675 owner: scheduled_nmordech@soko04.front.sepia.ceph.com priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 1255 sha1: 2fc421c83f0b86ea21b9e4289cad0f7957e09eb1 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle-suite suite_path: /home/teuthworker/src/github.com_NitzanMordhai_ceph_624b9a7bcb2064ce17206482df6a1da007ff7675/qa suite_relpath: qa suite_repo: https://github.com/NitzanMordhai/ceph.git suite_sha1: 624b9a7bcb2064ce17206482df6a1da007ff7675 targets: trial144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCL0uTHEX1sSFbWwP7rqeP9D1+zNYjU8SKRWPvZvBbSrhhC+OsCqG93BAdOW/BK0/jmUjboOIIQY2Pf2QpieS9I= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-19_12:10:46 tube: trial user: nmordech verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-19T12:40:41.138 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_NitzanMordhai_ceph_624b9a7bcb2064ce17206482df6a1da007ff7675/qa; will attempt to use it 2026-02-19T12:40:41.138 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_NitzanMordhai_ceph_624b9a7bcb2064ce17206482df6a1da007ff7675/qa/tasks 2026-02-19T12:40:41.138 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-19T12:40:41.140 INFO:teuthology.task.internal:Checking packages... 2026-02-19T12:40:41.140 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '2fc421c83f0b86ea21b9e4289cad0f7957e09eb1' 2026-02-19T12:40:41.140 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-19T12:40:41.140 INFO:teuthology.packaging:ref: None 2026-02-19T12:40:41.140 INFO:teuthology.packaging:tag: None 2026-02-19T12:40:41.140 INFO:teuthology.packaging:branch: wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle 2026-02-19T12:40:41.140 INFO:teuthology.packaging:sha1: 2fc421c83f0b86ea21b9e4289cad0f7957e09eb1 2026-02-19T12:40:41.140 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle 2026-02-19T12:40:41.261 INFO:teuthology.task.internal:Found packages for ceph version 20.2.0-720-g2fc421c8-1jammy 2026-02-19T12:40:41.262 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-19T12:40:41.271 INFO:teuthology.task.internal:no buildpackages task found 2026-02-19T12:40:41.271 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-19T12:40:41.283 INFO:teuthology.task.internal:Saving configuration 2026-02-19T12:40:41.291 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-19T12:40:41.303 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-19T12:40:41.380 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial144.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/nmordech-2026-02-19_12:10:46-rados-wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle-distro-default-trial/58618', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-19 12:37:12.084837', 'locked_by': 'scheduled_nmordech@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICb7lt/TPMM26DcF9xE29EwltyzMqod+Ub7gAwv89s6A'} 2026-02-19T12:40:41.380 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-19T12:40:41.391 INFO:teuthology.task.internal:roles: ubuntu@trial144.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-19T12:40:41.391 INFO:teuthology.run_tasks:Running task console_log... 2026-02-19T12:40:41.474 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7819c7d14f40>, signals=[15]) 2026-02-19T12:40:41.474 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-19T12:40:41.484 INFO:teuthology.task.internal:Opening connections... 2026-02-19T12:40:41.484 DEBUG:teuthology.task.internal:connecting to ubuntu@trial144.front.sepia.ceph.com 2026-02-19T12:40:41.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-19T12:40:41.551 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-19T12:40:41.562 DEBUG:teuthology.orchestra.run.trial144:> uname -m 2026-02-19T12:40:41.629 INFO:teuthology.orchestra.run.trial144.stdout:x86_64 2026-02-19T12:40:41.630 DEBUG:teuthology.orchestra.run.trial144:> cat /etc/os-release 2026-02-19T12:40:41.677 INFO:teuthology.orchestra.run.trial144.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-19T12:40:41.677 INFO:teuthology.orchestra.run.trial144.stdout:NAME="Ubuntu" 2026-02-19T12:40:41.677 INFO:teuthology.orchestra.run.trial144.stdout:VERSION_ID="22.04" 2026-02-19T12:40:41.677 INFO:teuthology.orchestra.run.trial144.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-19T12:40:41.677 INFO:teuthology.orchestra.run.trial144.stdout:VERSION_CODENAME=jammy 2026-02-19T12:40:41.678 INFO:teuthology.orchestra.run.trial144.stdout:ID=ubuntu 2026-02-19T12:40:41.678 INFO:teuthology.orchestra.run.trial144.stdout:ID_LIKE=debian 2026-02-19T12:40:41.678 INFO:teuthology.orchestra.run.trial144.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-19T12:40:41.678 INFO:teuthology.orchestra.run.trial144.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-19T12:40:41.678 INFO:teuthology.orchestra.run.trial144.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-19T12:40:41.678 INFO:teuthology.orchestra.run.trial144.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-19T12:40:41.678 INFO:teuthology.orchestra.run.trial144.stdout:UBUNTU_CODENAME=jammy 2026-02-19T12:40:41.678 INFO:teuthology.lock.ops:Updating trial144.front.sepia.ceph.com on lock server 2026-02-19T12:40:41.752 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-19T12:40:41.765 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-19T12:40:41.776 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-19T12:40:41.776 DEBUG:teuthology.orchestra.run.trial144:> test '!' -e /home/ubuntu/cephtest 2026-02-19T12:40:41.780 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-19T12:40:41.789 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-19T12:40:41.789 DEBUG:teuthology.orchestra.run.trial144:> test -z $(ls -A /var/lib/ceph) 2026-02-19T12:40:41.829 INFO:teuthology.orchestra.run.trial144.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-19T12:40:41.829 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-19T12:40:41.914 INFO:teuthology.run_tasks:Running task kernel... 2026-02-19T12:40:41.934 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-19T12:40:41.934 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-19T12:40:41.934 DEBUG:teuthology.orchestra.run.trial144:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-19T12:40:41.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T12:40:41.937 DEBUG:teuthology.orchestra.run.trial144:> uname -r 2026-02-19T12:40:41.985 INFO:teuthology.orchestra.run.trial144.stdout:5.15.0-170-generic 2026-02-19T12:40:41.985 INFO:teuthology.task.kernel:Running kernel on trial144: 5.15.0-170-generic 2026-02-19T12:40:41.985 DEBUG:teuthology.orchestra.run.trial144:> sudo apt-get clean 2026-02-19T12:40:42.048 DEBUG:teuthology.orchestra.run.trial144:> sudo apt-get update 2026-02-19T12:40:42.231 INFO:teuthology.orchestra.run.trial144.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-19T12:40:42.239 INFO:teuthology.orchestra.run.trial144.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-19T12:40:42.276 INFO:teuthology.orchestra.run.trial144.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-19T12:40:42.291 INFO:teuthology.orchestra.run.trial144.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-19T12:40:42.375 INFO:teuthology.orchestra.run.trial144.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3244 kB] 2026-02-19T12:40:42.440 INFO:teuthology.orchestra.run.trial144.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [495 kB] 2026-02-19T12:40:42.443 INFO:teuthology.orchestra.run.trial144.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.1 kB] 2026-02-19T12:40:42.443 INFO:teuthology.orchestra.run.trial144.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-19T12:40:42.451 INFO:teuthology.orchestra.run.trial144.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-19T12:40:42.453 INFO:teuthology.orchestra.run.trial144.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.3 kB] 2026-02-19T12:40:42.453 INFO:teuthology.orchestra.run.trial144.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5310 kB] 2026-02-19T12:40:42.482 INFO:teuthology.orchestra.run.trial144.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1005 kB] 2026-02-19T12:40:42.484 INFO:teuthology.orchestra.run.trial144.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2983 kB] 2026-02-19T12:40:42.493 INFO:teuthology.orchestra.run.trial144.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [427 kB] 2026-02-19T12:40:42.497 INFO:teuthology.orchestra.run.trial144.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.7 kB] 2026-02-19T12:40:42.497 INFO:teuthology.orchestra.run.trial144.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5147 kB] 2026-02-19T12:40:42.513 INFO:teuthology.orchestra.run.trial144.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [979 kB] 2026-02-19T12:40:46.366 INFO:teuthology.orchestra.run.trial144.stdout:Fetched 21.6 MB in 2s (13.6 MB/s) 2026-02-19T12:40:46.889 INFO:teuthology.orchestra.run.trial144.stdout:Reading package lists... 2026-02-19T12:40:46.903 DEBUG:teuthology.orchestra.run.trial144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-19T12:40:46.935 INFO:teuthology.orchestra.run.trial144.stdout:Reading package lists... 2026-02-19T12:40:47.042 INFO:teuthology.orchestra.run.trial144.stdout:Building dependency tree... 2026-02-19T12:40:47.043 INFO:teuthology.orchestra.run.trial144.stdout:Reading state information... 2026-02-19T12:40:47.125 INFO:teuthology.orchestra.run.trial144.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-19T12:40:47.125 INFO:teuthology.orchestra.run.trial144.stdout:linux-image-generic set to manually installed. 2026-02-19T12:40:47.126 INFO:teuthology.orchestra.run.trial144.stdout:The following packages were automatically installed and are no longer required: 2026-02-19T12:40:47.126 INFO:teuthology.orchestra.run.trial144.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-19T12:40:47.126 INFO:teuthology.orchestra.run.trial144.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-19T12:40:47.126 INFO:teuthology.orchestra.run.trial144.stdout: sg3-utils-udev 2026-02-19T12:40:47.126 INFO:teuthology.orchestra.run.trial144.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-19T12:40:47.144 INFO:teuthology.orchestra.run.trial144.stdout:0 upgraded, 0 newly installed, 0 to remove and 12 not upgraded. 2026-02-19T12:40:47.146 DEBUG:teuthology.orchestra.run.trial144:> dpkg -s linux-image-generic 2026-02-19T12:40:47.199 INFO:teuthology.orchestra.run.trial144.stdout:Package: linux-image-generic 2026-02-19T12:40:47.199 INFO:teuthology.orchestra.run.trial144.stdout:Status: install ok installed 2026-02-19T12:40:47.199 INFO:teuthology.orchestra.run.trial144.stdout:Priority: optional 2026-02-19T12:40:47.199 INFO:teuthology.orchestra.run.trial144.stdout:Section: kernel 2026-02-19T12:40:47.199 INFO:teuthology.orchestra.run.trial144.stdout:Installed-Size: 21 2026-02-19T12:40:47.200 INFO:teuthology.orchestra.run.trial144.stdout:Maintainer: Ubuntu Kernel Team 2026-02-19T12:40:47.200 INFO:teuthology.orchestra.run.trial144.stdout:Architecture: amd64 2026-02-19T12:40:47.200 INFO:teuthology.orchestra.run.trial144.stdout:Source: linux-meta 2026-02-19T12:40:47.200 INFO:teuthology.orchestra.run.trial144.stdout:Version: 5.15.0.170.159 2026-02-19T12:40:47.200 INFO:teuthology.orchestra.run.trial144.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-19T12:40:47.200 INFO:teuthology.orchestra.run.trial144.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-19T12:40:47.200 INFO:teuthology.orchestra.run.trial144.stdout:Recommends: thermald 2026-02-19T12:40:47.200 INFO:teuthology.orchestra.run.trial144.stdout:Description: Generic Linux kernel image 2026-02-19T12:40:47.200 INFO:teuthology.orchestra.run.trial144.stdout: This package will always depend on the latest generic kernel image 2026-02-19T12:40:47.200 INFO:teuthology.orchestra.run.trial144.stdout: available. 2026-02-19T12:40:47.201 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-19T12:40:47.201 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-19T12:40:47.201 DEBUG:teuthology.orchestra.run.trial144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-19T12:40:47.249 INFO:teuthology.orchestra.run.trial144.stdout:ttyS1 2026-02-19T12:40:47.280 DEBUG:teuthology.parallel:result is None 2026-02-19T12:40:47.280 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-19T12:40:47.291 INFO:teuthology.task.internal:Creating test directory... 2026-02-19T12:40:47.291 DEBUG:teuthology.orchestra.run.trial144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-19T12:40:47.325 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-19T12:40:47.338 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-19T12:40:47.357 INFO:teuthology.task.internal:Creating archive directory... 2026-02-19T12:40:47.357 DEBUG:teuthology.orchestra.run.trial144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-19T12:40:47.371 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-19T12:40:47.381 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-19T12:40:47.381 DEBUG:teuthology.orchestra.run.trial144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-19T12:40:47.417 INFO:teuthology.orchestra.run.trial144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:40:47.424 INFO:teuthology.orchestra.run.trial144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:40:47.425 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-19T12:40:47.436 INFO:teuthology.task.internal:Configuring sudo... 2026-02-19T12:40:47.436 DEBUG:teuthology.orchestra.run.trial144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-19T12:40:47.473 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-19T12:40:47.486 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-19T12:40:47.486 DEBUG:teuthology.orchestra.run.trial144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-19T12:40:47.517 DEBUG:teuthology.orchestra.run.trial144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-19T12:40:47.560 DEBUG:teuthology.orchestra.run.trial144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-19T12:40:47.604 DEBUG:teuthology.orchestra.run.trial144:> set -ex 2026-02-19T12:40:47.604 DEBUG:teuthology.orchestra.run.trial144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-19T12:40:47.654 DEBUG:teuthology.orchestra.run.trial144:> sudo service rsyslog restart 2026-02-19T12:40:47.714 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-19T12:40:47.728 INFO:teuthology.task.internal:Starting timer... 2026-02-19T12:40:47.728 INFO:teuthology.run_tasks:Running task pcp... 2026-02-19T12:40:47.746 INFO:teuthology.run_tasks:Running task selinux... 2026-02-19T12:40:47.758 DEBUG:teuthology.task.selinux:Excluding trial144: OS 'ubuntu' does not support SELinux 2026-02-19T12:40:47.758 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-19T12:40:47.758 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-19T12:40:47.759 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-19T12:40:47.759 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-19T12:40:47.770 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-19T12:40:47.779 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-19T12:40:47.779 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-19T12:41:47.242 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial144.front.sepia.ceph.com')] 2026-02-19T12:41:47.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial144.front.sepia.ceph.com' 2026-02-19T12:41:47.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-19T12:41:47.355 DEBUG:teuthology.orchestra.run.trial144:> true 2026-02-19T12:41:47.600 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial144.front.sepia.ceph.com' 2026-02-19T12:41:47.600 INFO:teuthology.run_tasks:Running task clock... 2026-02-19T12:41:47.612 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-19T12:41:47.612 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-19T12:41:47.613 DEBUG:teuthology.orchestra.run.trial144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-19T12:41:47.659 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-19T12:41:47.659 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: Command line: ntpd -gq 2026-02-19T12:41:47.659 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: ---------------------------------------------------- 2026-02-19T12:41:47.659 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: ntp-4 is maintained by Network Time Foundation, 2026-02-19T12:41:47.660 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-19T12:41:47.660 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: corporation. Support and training for ntp-4 are 2026-02-19T12:41:47.660 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: available at https://www.nwtime.org/support 2026-02-19T12:41:47.660 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: ---------------------------------------------------- 2026-02-19T12:41:47.660 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: proto: precision = 0.020 usec (-25) 2026-02-19T12:41:47.660 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: basedate set to 2022-02-04 2026-02-19T12:41:47.660 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: gps base set to 2022-02-06 (week 2196) 2026-02-19T12:41:47.660 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-19T12:41:47.661 INFO:teuthology.orchestra.run.trial144.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-19T12:41:47.661 INFO:teuthology.orchestra.run.trial144.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-19T12:41:47.662 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: restrict ::: KOD does nothing without LIMITED. 2026-02-19T12:41:47.662 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: Listen and drop on 0 v6wildcard [::]:123 2026-02-19T12:41:47.662 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-19T12:41:47.662 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: Listen normally on 2 lo 127.0.0.1:123 2026-02-19T12:41:47.662 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: Listen normally on 3 enp1s0f0 10.20.193.144:123 2026-02-19T12:41:47.662 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: Listen normally on 4 lo [::1]:123 2026-02-19T12:41:47.662 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:61d6%2]:123 2026-02-19T12:41:47.662 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:47 ntpd[5488]: Listening on routing socket on fd #22 for interface updates 2026-02-19T12:41:48.659 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:48 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:48.659 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:48 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:48.659 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:48 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:48.659 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:48 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:48.663 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:48 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:48.663 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:48 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:48.663 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:48 ntpd[5488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-19T12:41:48.663 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:48 ntpd[5488]: can't open /var/log/ntpstats/peerstats.20260219: Permission denied 2026-02-19T12:41:48.729 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:48 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:48.730 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:48 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:48.730 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:48 ntpd[5488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-19T12:41:48.730 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:48 ntpd[5488]: can't open /var/log/ntpstats/peerstats.20260219: Permission denied 2026-02-19T12:41:49.658 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:49 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:49.659 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:49 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:49.724 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:49 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:49.724 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:49 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:49.724 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:49 ntpd[5488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-19T12:41:49.724 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:49 ntpd[5488]: can't open /var/log/ntpstats/peerstats.20260219: Permission denied 2026-02-19T12:41:50.659 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:50.659 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:50.660 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:50.660 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:50.660 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:50.660 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:50.663 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:50.663 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:50.663 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-19T12:41:50.663 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: can't open /var/log/ntpstats/peerstats.20260219: Permission denied 2026-02-19T12:41:50.712 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:50.712 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:50.712 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-19T12:41:50.712 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: can't open /var/log/ntpstats/peerstats.20260219: Permission denied 2026-02-19T12:41:50.730 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:50.730 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:50.730 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-19T12:41:50.730 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:50 ntpd[5488]: can't open /var/log/ntpstats/peerstats.20260219: Permission denied 2026-02-19T12:41:51.659 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:51 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:51.659 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:51 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:51.723 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:51 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:51.723 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:51 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:51.723 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:51 ntpd[5488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-19T12:41:51.723 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:51 ntpd[5488]: can't open /var/log/ntpstats/peerstats.20260219: Permission denied 2026-02-19T12:41:52.660 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:52.660 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:52.660 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:52.661 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:52.661 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:52.661 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:52.663 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:52.663 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:52.663 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-19T12:41:52.663 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: can't open /var/log/ntpstats/peerstats.20260219: Permission denied 2026-02-19T12:41:52.711 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:52.712 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:52.712 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-19T12:41:52.712 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: can't open /var/log/ntpstats/peerstats.20260219: Permission denied 2026-02-19T12:41:52.729 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:52.730 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:52.730 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-19T12:41:52.730 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:52 ntpd[5488]: can't open /var/log/ntpstats/peerstats.20260219: Permission denied 2026-02-19T12:41:53.659 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:53 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:53.659 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:53 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:53.723 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:53 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:53.723 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:53 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:53.723 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:53 ntpd[5488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-19T12:41:53.724 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:53 ntpd[5488]: can't open /var/log/ntpstats/peerstats.20260219: Permission denied 2026-02-19T12:41:54.659 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:54 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:54.659 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:54 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:54.660 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:54 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:54.660 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:54 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:54.660 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:54 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:54.660 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:54 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:54.662 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:54 ntpd[5488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-19T12:41:54.663 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:54 ntpd[5488]: can't open /var/log/ntpstats/rawstats.20260219: Permission denied 2026-02-19T12:41:54.663 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:54 ntpd[5488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-19T12:41:54.663 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:54 ntpd[5488]: can't open /var/log/ntpstats/peerstats.20260219: Permission denied 2026-02-19T12:41:54.663 INFO:teuthology.orchestra.run.trial144.stdout:19 Feb 12:41:54 ntpd[5488]: ntpd: time set +0.047537 s 2026-02-19T12:41:54.663 INFO:teuthology.orchestra.run.trial144.stdout:ntpd: time set +0.047537s 2026-02-19T12:41:54.664 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:54 ntpd[5488]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-19T12:41:54.664 INFO:teuthology.orchestra.run.trial144.stderr:19 Feb 12:41:54 ntpd[5488]: can't open /var/log/ntpstats/loopstats.20260219: Permission denied 2026-02-19T12:41:54.708 INFO:teuthology.orchestra.run.trial144.stdout: remote refid st t when poll reach delay offset jitter 2026-02-19T12:41:54.708 INFO:teuthology.orchestra.run.trial144.stdout:============================================================================== 2026-02-19T12:41:54.708 INFO:teuthology.orchestra.run.trial144.stdout: hou1.us.ntp.li .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-19T12:41:54.708 INFO:teuthology.orchestra.run.trial144.stdout: 96-19-94-82.cpe .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-19T12:41:54.709 INFO:teuthology.run_tasks:Running task install... 2026-02-19T12:41:54.719 DEBUG:teuthology.task.install:project ceph 2026-02-19T12:41:54.719 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2fc421c83f0b86ea21b9e4289cad0f7957e09eb1'}} 2026-02-19T12:41:54.719 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '2fc421c83f0b86ea21b9e4289cad0f7957e09eb1'} 2026-02-19T12:41:54.719 INFO:teuthology.task.install:Using flavor: default 2026-02-19T12:41:54.723 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-19T12:41:54.724 INFO:teuthology.task.install:extra packages: [] 2026-02-19T12:41:54.724 DEBUG:teuthology.orchestra.run.trial144:> sudo apt-key list | grep Ceph 2026-02-19T12:41:54.796 INFO:teuthology.orchestra.run.trial144.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-19T12:41:54.825 INFO:teuthology.orchestra.run.trial144.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-19T12:41:54.825 INFO:teuthology.orchestra.run.trial144.stdout:uid [ unknown] Ceph.com (release key) 2026-02-19T12:41:54.826 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-19T12:41:54.827 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2fc421c83f0b86ea21b9e4289cad0f7957e09eb1 2026-02-19T12:41:54.991 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default/ 2026-02-19T12:41:54.991 INFO:teuthology.task.install.deb:Package version is 20.2.0-720-g2fc421c8-1jammy 2026-02-19T12:41:55.188 DEBUG:teuthology.orchestra.run.trial144:> set -ex 2026-02-19T12:41:55.188 DEBUG:teuthology.orchestra.run.trial144:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-19T12:41:55.198 DEBUG:teuthology.orchestra.run.trial144:> sudo apt-get update 2026-02-19T12:41:55.365 INFO:teuthology.orchestra.run.trial144.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy InRelease 2026-02-19T12:41:55.377 INFO:teuthology.orchestra.run.trial144.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy Release [7680 B] 2026-02-19T12:41:55.388 INFO:teuthology.orchestra.run.trial144.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-19T12:41:55.400 INFO:teuthology.orchestra.run.trial144.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.9 kB] 2026-02-19T12:41:55.634 INFO:teuthology.orchestra.run.trial144.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-19T12:41:55.705 INFO:teuthology.orchestra.run.trial144.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-19T12:41:55.776 INFO:teuthology.orchestra.run.trial144.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-19T12:41:55.849 INFO:teuthology.orchestra.run.trial144.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-19T12:41:55.989 INFO:teuthology.orchestra.run.trial144.stdout:Fetched 26.5 kB in 1s (42.1 kB/s) 2026-02-19T12:41:56.522 INFO:teuthology.orchestra.run.trial144.stdout:Reading package lists... 2026-02-19T12:41:56.536 DEBUG:teuthology.orchestra.run.trial144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=20.2.0-720-g2fc421c8-1jammy cephadm=20.2.0-720-g2fc421c8-1jammy ceph-mds=20.2.0-720-g2fc421c8-1jammy ceph-mgr=20.2.0-720-g2fc421c8-1jammy ceph-common=20.2.0-720-g2fc421c8-1jammy ceph-fuse=20.2.0-720-g2fc421c8-1jammy ceph-test=20.2.0-720-g2fc421c8-1jammy ceph-volume=20.2.0-720-g2fc421c8-1jammy radosgw=20.2.0-720-g2fc421c8-1jammy python3-rados=20.2.0-720-g2fc421c8-1jammy python3-rgw=20.2.0-720-g2fc421c8-1jammy python3-cephfs=20.2.0-720-g2fc421c8-1jammy python3-rbd=20.2.0-720-g2fc421c8-1jammy libcephfs2=20.2.0-720-g2fc421c8-1jammy libcephfs-dev=20.2.0-720-g2fc421c8-1jammy librados2=20.2.0-720-g2fc421c8-1jammy librbd1=20.2.0-720-g2fc421c8-1jammy rbd-fuse=20.2.0-720-g2fc421c8-1jammy 2026-02-19T12:41:56.570 INFO:teuthology.orchestra.run.trial144.stdout:Reading package lists... 2026-02-19T12:41:56.679 INFO:teuthology.orchestra.run.trial144.stdout:Building dependency tree... 2026-02-19T12:41:56.680 INFO:teuthology.orchestra.run.trial144.stdout:Reading state information... 2026-02-19T12:41:56.773 INFO:teuthology.orchestra.run.trial144.stdout:The following packages were automatically installed and are no longer required: 2026-02-19T12:41:56.773 INFO:teuthology.orchestra.run.trial144.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-19T12:41:56.774 INFO:teuthology.orchestra.run.trial144.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-19T12:41:56.774 INFO:teuthology.orchestra.run.trial144.stdout: sg3-utils-udev 2026-02-19T12:41:56.774 INFO:teuthology.orchestra.run.trial144.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-19T12:41:56.774 INFO:teuthology.orchestra.run.trial144.stdout:The following additional packages will be installed: 2026-02-19T12:41:56.774 INFO:teuthology.orchestra.run.trial144.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-19T12:41:56.774 INFO:teuthology.orchestra.run.trial144.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2026-02-19T12:41:56.775 INFO:teuthology.orchestra.run.trial144.stdout: libcephfs-daemon libcephfs-proxy2 libdouble-conversion3 libfuse2 libjq1 2026-02-19T12:41:56.775 INFO:teuthology.orchestra.run.trial144.stdout: liblttng-ust1 libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2026-02-19T12:41:56.775 INFO:teuthology.orchestra.run.trial144.stdout: libqt5dbus5 libqt5network5 libradosstriper1 librdkafka1 librgw2 2026-02-19T12:41:56.775 INFO:teuthology.orchestra.run.trial144.stdout: libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli python-asyncssh-doc 2026-02-19T12:41:56.775 INFO:teuthology.orchestra.run.trial144.stdout: python3-asyncssh python3-cachetools python3-ceph-argparse 2026-02-19T12:41:56.775 INFO:teuthology.orchestra.run.trial144.stdout: python3-ceph-common python3-cheroot python3-cherrypy3 python3-google-auth 2026-02-19T12:41:56.776 INFO:teuthology.orchestra.run.trial144.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2026-02-19T12:41:56.776 INFO:teuthology.orchestra.run.trial144.stdout: python3-jaraco.functools python3-jaraco.text python3-joblib 2026-02-19T12:41:56.776 INFO:teuthology.orchestra.run.trial144.stdout: python3-kubernetes python3-natsort python3-pluggy python3-portend 2026-02-19T12:41:56.776 INFO:teuthology.orchestra.run.trial144.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-19T12:41:56.776 INFO:teuthology.orchestra.run.trial144.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2026-02-19T12:41:56.776 INFO:teuthology.orchestra.run.trial144.stdout: python3-rsa python3-simplejson python3-sklearn python3-sklearn-lib 2026-02-19T12:41:56.776 INFO:teuthology.orchestra.run.trial144.stdout: python3-tempora python3-threadpoolctl python3-toml python3-wcwidth 2026-02-19T12:41:56.776 INFO:teuthology.orchestra.run.trial144.stdout: python3-webob python3-websocket python3-zc.lockfile qttranslations5-l10n 2026-02-19T12:41:56.776 INFO:teuthology.orchestra.run.trial144.stdout: socat xmlstarlet 2026-02-19T12:41:56.776 INFO:teuthology.orchestra.run.trial144.stdout:Suggested packages: 2026-02-19T12:41:56.777 INFO:teuthology.orchestra.run.trial144.stdout: python3-influxdb liblua5.3-dev luarocks python-natsort-doc python-psutil-doc 2026-02-19T12:41:56.777 INFO:teuthology.orchestra.run.trial144.stdout: subversion python-pygments-doc ttf-bitstream-vera python3-paste python3-dap 2026-02-19T12:41:56.777 INFO:teuthology.orchestra.run.trial144.stdout: python-sklearn-doc ipython3 python-webob-doc 2026-02-19T12:41:56.777 INFO:teuthology.orchestra.run.trial144.stdout:Recommended packages: 2026-02-19T12:41:56.777 INFO:teuthology.orchestra.run.trial144.stdout: btrfs-tools 2026-02-19T12:41:56.806 INFO:teuthology.orchestra.run.trial144.stdout:The following NEW packages will be installed: 2026-02-19T12:41:56.807 INFO:teuthology.orchestra.run.trial144.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-19T12:41:56.807 INFO:teuthology.orchestra.run.trial144.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-19T12:41:56.807 INFO:teuthology.orchestra.run.trial144.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2026-02-19T12:41:56.807 INFO:teuthology.orchestra.run.trial144.stdout: libcephfs-daemon libcephfs-dev libcephfs-proxy2 libcephfs2 2026-02-19T12:41:56.807 INFO:teuthology.orchestra.run.trial144.stdout: libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 liboath0 2026-02-19T12:41:56.807 INFO:teuthology.orchestra.run.trial144.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2026-02-19T12:41:56.807 INFO:teuthology.orchestra.run.trial144.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2026-02-19T12:41:56.807 INFO:teuthology.orchestra.run.trial144.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python3-asyncssh 2026-02-19T12:41:56.807 INFO:teuthology.orchestra.run.trial144.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2026-02-19T12:41:56.807 INFO:teuthology.orchestra.run.trial144.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-19T12:41:56.808 INFO:teuthology.orchestra.run.trial144.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-19T12:41:56.808 INFO:teuthology.orchestra.run.trial144.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-natsort 2026-02-19T12:41:56.808 INFO:teuthology.orchestra.run.trial144.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2026-02-19T12:41:56.808 INFO:teuthology.orchestra.run.trial144.stdout: python3-pygments python3-pytest python3-rados python3-rbd python3-repoze.lru 2026-02-19T12:41:56.808 INFO:teuthology.orchestra.run.trial144.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2026-02-19T12:41:56.808 INFO:teuthology.orchestra.run.trial144.stdout: python3-simplejson python3-sklearn python3-sklearn-lib python3-tempora 2026-02-19T12:41:56.808 INFO:teuthology.orchestra.run.trial144.stdout: python3-threadpoolctl python3-toml python3-wcwidth python3-webob 2026-02-19T12:41:56.808 INFO:teuthology.orchestra.run.trial144.stdout: python3-websocket python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2026-02-19T12:41:56.808 INFO:teuthology.orchestra.run.trial144.stdout: socat xmlstarlet 2026-02-19T12:41:56.939 INFO:teuthology.orchestra.run.trial144.stdout:0 upgraded, 86 newly installed, 0 to remove and 12 not upgraded. 2026-02-19T12:41:56.939 INFO:teuthology.orchestra.run.trial144.stdout:Need to get 280 MB of archives. 2026-02-19T12:41:56.940 INFO:teuthology.orchestra.run.trial144.stdout:After this operation, 1119 MB of additional disk space will be used. 2026-02-19T12:41:56.940 INFO:teuthology.orchestra.run.trial144.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 20.2.0-720-g2fc421c8-1jammy [3569 kB] 2026-02-19T12:41:57.237 INFO:teuthology.orchestra.run.trial144.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-19T12:41:57.285 INFO:teuthology.orchestra.run.trial144.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 20.2.0-720-g2fc421c8-1jammy [2867 kB] 2026-02-19T12:41:57.364 INFO:teuthology.orchestra.run.trial144.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 20.2.0-720-g2fc421c8-1jammy [833 kB] 2026-02-19T12:41:57.385 INFO:teuthology.orchestra.run.trial144.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 20.2.0-720-g2fc421c8-1jammy [365 kB] 2026-02-19T12:41:57.395 INFO:teuthology.orchestra.run.trial144.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 20.2.0-720-g2fc421c8-1jammy [32.9 kB] 2026-02-19T12:41:57.395 INFO:teuthology.orchestra.run.trial144.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 20.2.0-720-g2fc421c8-1jammy [184 kB] 2026-02-19T12:41:57.396 INFO:teuthology.orchestra.run.trial144.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 20.2.0-720-g2fc421c8-1jammy [84.0 kB] 2026-02-19T12:41:57.397 INFO:teuthology.orchestra.run.trial144.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 20.2.0-720-g2fc421c8-1jammy [341 kB] 2026-02-19T12:41:57.409 INFO:teuthology.orchestra.run.trial144.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 20.2.0-720-g2fc421c8-1jammy [8693 kB] 2026-02-19T12:41:57.646 INFO:teuthology.orchestra.run.trial144.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-19T12:41:57.659 INFO:teuthology.orchestra.run.trial144.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-19T12:41:57.702 INFO:teuthology.orchestra.run.trial144.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 20.2.0-720-g2fc421c8-1jammy [112 kB] 2026-02-19T12:41:57.703 INFO:teuthology.orchestra.run.trial144.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 20.2.0-720-g2fc421c8-1jammy [261 kB] 2026-02-19T12:41:57.707 INFO:teuthology.orchestra.run.trial144.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 20.2.0-720-g2fc421c8-1jammy [29.3 MB] 2026-02-19T12:41:57.744 INFO:teuthology.orchestra.run.trial144.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-19T12:41:57.990 INFO:teuthology.orchestra.run.trial144.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-19T12:41:58.002 INFO:teuthology.orchestra.run.trial144.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-19T12:41:58.034 INFO:teuthology.orchestra.run.trial144.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-19T12:41:58.044 INFO:teuthology.orchestra.run.trial144.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-19T12:41:58.047 INFO:teuthology.orchestra.run.trial144.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-19T12:41:58.048 INFO:teuthology.orchestra.run.trial144.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-19T12:41:58.049 INFO:teuthology.orchestra.run.trial144.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-19T12:41:58.069 INFO:teuthology.orchestra.run.trial144.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-19T12:41:58.070 INFO:teuthology.orchestra.run.trial144.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-19T12:41:58.077 INFO:teuthology.orchestra.run.trial144.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-19T12:41:58.159 INFO:teuthology.orchestra.run.trial144.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-19T12:41:58.159 INFO:teuthology.orchestra.run.trial144.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-19T12:41:58.159 INFO:teuthology.orchestra.run.trial144.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-19T12:41:58.160 INFO:teuthology.orchestra.run.trial144.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-19T12:41:58.160 INFO:teuthology.orchestra.run.trial144.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-19T12:41:58.160 INFO:teuthology.orchestra.run.trial144.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-19T12:41:58.160 INFO:teuthology.orchestra.run.trial144.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-19T12:41:58.166 INFO:teuthology.orchestra.run.trial144.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-19T12:41:58.166 INFO:teuthology.orchestra.run.trial144.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-19T12:41:58.251 INFO:teuthology.orchestra.run.trial144.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-19T12:41:58.254 INFO:teuthology.orchestra.run.trial144.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-19T12:41:58.254 INFO:teuthology.orchestra.run.trial144.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-19T12:41:58.255 INFO:teuthology.orchestra.run.trial144.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-19T12:41:58.285 INFO:teuthology.orchestra.run.trial144.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-19T12:41:58.287 INFO:teuthology.orchestra.run.trial144.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-19T12:41:58.288 INFO:teuthology.orchestra.run.trial144.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-19T12:41:58.358 INFO:teuthology.orchestra.run.trial144.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-19T12:41:58.359 INFO:teuthology.orchestra.run.trial144.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-19T12:41:58.359 INFO:teuthology.orchestra.run.trial144.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-19T12:41:58.359 INFO:teuthology.orchestra.run.trial144.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-19T12:41:58.360 INFO:teuthology.orchestra.run.trial144.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-19T12:41:58.360 INFO:teuthology.orchestra.run.trial144.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-19T12:41:58.433 INFO:teuthology.orchestra.run.trial144.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-19T12:41:58.434 INFO:teuthology.orchestra.run.trial144.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-19T12:41:58.436 INFO:teuthology.orchestra.run.trial144.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-19T12:41:58.521 INFO:teuthology.orchestra.run.trial144.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-19T12:41:58.524 INFO:teuthology.orchestra.run.trial144.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-19T12:41:58.527 INFO:teuthology.orchestra.run.trial144.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-19T12:41:58.531 INFO:teuthology.orchestra.run.trial144.stdout:Get:55 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-19T12:41:58.534 INFO:teuthology.orchestra.run.trial144.stdout:Get:56 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-19T12:41:58.535 INFO:teuthology.orchestra.run.trial144.stdout:Get:57 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-19T12:41:58.535 INFO:teuthology.orchestra.run.trial144.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-19T12:41:58.608 INFO:teuthology.orchestra.run.trial144.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-19T12:41:58.609 INFO:teuthology.orchestra.run.trial144.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-19T12:41:58.616 INFO:teuthology.orchestra.run.trial144.stdout:Get:61 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-19T12:41:58.696 INFO:teuthology.orchestra.run.trial144.stdout:Get:62 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-19T12:41:58.698 INFO:teuthology.orchestra.run.trial144.stdout:Get:63 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-19T12:41:58.699 INFO:teuthology.orchestra.run.trial144.stdout:Get:64 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-19T12:41:58.700 INFO:teuthology.orchestra.run.trial144.stdout:Get:65 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-19T12:41:59.047 INFO:teuthology.orchestra.run.trial144.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 20.2.0-720-g2fc421c8-1jammy [5416 kB] 2026-02-19T12:41:59.297 INFO:teuthology.orchestra.run.trial144.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 20.2.0-720-g2fc421c8-1jammy [250 kB] 2026-02-19T12:41:59.306 INFO:teuthology.orchestra.run.trial144.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 20.2.0-720-g2fc421c8-1jammy [124 kB] 2026-02-19T12:41:59.311 INFO:teuthology.orchestra.run.trial144.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 20.2.0-720-g2fc421c8-1jammy [911 kB] 2026-02-19T12:41:59.355 INFO:teuthology.orchestra.run.trial144.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 20.2.0-720-g2fc421c8-1jammy [6398 kB] 2026-02-19T12:41:59.661 INFO:teuthology.orchestra.run.trial144.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 20.2.0-720-g2fc421c8-1jammy [21.7 MB] 2026-02-19T12:42:00.685 INFO:teuthology.orchestra.run.trial144.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 20.2.0-720-g2fc421c8-1jammy [14.1 kB] 2026-02-19T12:42:00.685 INFO:teuthology.orchestra.run.trial144.stdout:Get:73 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 20.2.0-720-g2fc421c8-1jammy [957 kB] 2026-02-19T12:42:00.728 INFO:teuthology.orchestra.run.trial144.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 20.2.0-720-g2fc421c8-1jammy [2344 kB] 2026-02-19T12:42:00.839 INFO:teuthology.orchestra.run.trial144.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 20.2.0-720-g2fc421c8-1jammy [1049 kB] 2026-02-19T12:42:00.889 INFO:teuthology.orchestra.run.trial144.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 20.2.0-720-g2fc421c8-1jammy [179 kB] 2026-02-19T12:42:00.896 INFO:teuthology.orchestra.run.trial144.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 20.2.0-720-g2fc421c8-1jammy [45.5 MB] 2026-02-19T12:42:02.674 INFO:teuthology.orchestra.run.trial144.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 20.2.0-720-g2fc421c8-1jammy [8626 kB] 2026-02-19T12:42:03.054 INFO:teuthology.orchestra.run.trial144.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 20.2.0-720-g2fc421c8-1jammy [14.2 kB] 2026-02-19T12:42:03.055 INFO:teuthology.orchestra.run.trial144.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 20.2.0-720-g2fc421c8-1jammy [99.5 MB] 2026-02-19T12:42:07.017 INFO:teuthology.orchestra.run.trial144.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 20.2.0-720-g2fc421c8-1jammy [135 kB] 2026-02-19T12:42:07.017 INFO:teuthology.orchestra.run.trial144.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-daemon amd64 20.2.0-720-g2fc421c8-1jammy [43.2 kB] 2026-02-19T12:42:07.018 INFO:teuthology.orchestra.run.trial144.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-proxy2 amd64 20.2.0-720-g2fc421c8-1jammy [30.7 kB] 2026-02-19T12:42:07.018 INFO:teuthology.orchestra.run.trial144.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 20.2.0-720-g2fc421c8-1jammy [41.5 kB] 2026-02-19T12:42:07.019 INFO:teuthology.orchestra.run.trial144.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 20.2.0-720-g2fc421c8-1jammy [25.1 MB] 2026-02-19T12:42:08.352 INFO:teuthology.orchestra.run.trial144.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 20.2.0-720-g2fc421c8-1jammy [97.5 kB] 2026-02-19T12:42:08.638 INFO:teuthology.orchestra.run.trial144.stdout:Fetched 280 MB in 12s (24.2 MB/s) 2026-02-19T12:42:08.654 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-19T12:42:08.808 INFO:teuthology.orchestra.run.trial144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167442 files and directories currently installed.) 2026-02-19T12:42:08.809 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../00-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-19T12:42:08.810 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-19T12:42:08.831 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-19T12:42:08.844 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../01-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-19T12:42:08.844 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-19T12:42:08.863 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-19T12:42:08.875 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../02-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-19T12:42:08.876 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-19T12:42:08.896 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-19T12:42:08.904 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../03-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-19T12:42:08.909 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-19T12:42:08.940 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-19T12:42:08.948 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../04-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-19T12:42:08.948 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-19T12:42:08.964 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-19T12:42:08.973 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../05-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-19T12:42:08.973 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-19T12:42:08.993 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-19T12:42:09.001 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../06-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-19T12:42:09.001 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-19T12:42:09.018 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package librados2. 2026-02-19T12:42:09.026 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../07-librados2_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:09.026 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking librados2 (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:09.068 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libnbd0. 2026-02-19T12:42:09.076 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../08-libnbd0_1.10.5-1_amd64.deb ... 2026-02-19T12:42:09.076 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-19T12:42:09.090 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package librbd1. 2026-02-19T12:42:09.098 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../09-librbd1_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:09.098 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking librbd1 (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:09.135 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libcephfs2. 2026-02-19T12:42:09.143 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../10-libcephfs2_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:09.144 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libcephfs2 (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:09.164 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-rados. 2026-02-19T12:42:09.172 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../11-python3-rados_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:09.173 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-rados (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:09.189 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-19T12:42:09.197 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../12-python3-ceph-argparse_20.2.0-720-g2fc421c8-1jammy_all.deb ... 2026-02-19T12:42:09.198 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-ceph-argparse (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:09.212 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-cephfs. 2026-02-19T12:42:09.220 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../13-python3-cephfs_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:09.220 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-cephfs (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:09.236 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-19T12:42:09.244 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../14-python3-ceph-common_20.2.0-720-g2fc421c8-1jammy_all.deb ... 2026-02-19T12:42:09.244 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-ceph-common (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:09.262 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-19T12:42:09.270 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../15-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-19T12:42:09.271 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-19T12:42:09.287 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-prettytable. 2026-02-19T12:42:09.295 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../16-python3-prettytable_2.5.0-2_all.deb ... 2026-02-19T12:42:09.296 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-19T12:42:09.310 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-rbd. 2026-02-19T12:42:09.318 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../17-python3-rbd_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:09.319 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-rbd (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:09.336 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-19T12:42:09.344 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../18-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-19T12:42:09.345 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-19T12:42:09.363 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package librgw2. 2026-02-19T12:42:09.371 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../19-librgw2_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:09.372 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking librgw2 (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:09.465 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-rgw. 2026-02-19T12:42:09.473 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../20-python3-rgw_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:09.474 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-rgw (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:09.490 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-19T12:42:09.498 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../21-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-19T12:42:09.498 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-19T12:42:09.512 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libradosstriper1. 2026-02-19T12:42:09.521 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../22-libradosstriper1_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:09.521 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libradosstriper1 (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:09.538 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package ceph-common. 2026-02-19T12:42:09.546 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../23-ceph-common_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:09.546 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking ceph-common (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:09.790 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package ceph-base. 2026-02-19T12:42:09.799 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../24-ceph-base_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:09.804 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking ceph-base (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:09.868 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-19T12:42:09.877 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../25-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-19T12:42:09.877 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-19T12:42:09.892 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-cheroot. 2026-02-19T12:42:09.900 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../26-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-19T12:42:09.901 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-19T12:42:09.918 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-19T12:42:09.926 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../27-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-19T12:42:09.927 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-19T12:42:09.941 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-19T12:42:09.949 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../28-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-19T12:42:09.950 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-19T12:42:09.964 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-19T12:42:09.972 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../29-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-19T12:42:09.973 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-19T12:42:09.987 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-tempora. 2026-02-19T12:42:09.996 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../30-python3-tempora_4.1.2-1_all.deb ... 2026-02-19T12:42:09.996 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-19T12:42:10.011 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-portend. 2026-02-19T12:42:10.020 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../31-python3-portend_3.0.0-1_all.deb ... 2026-02-19T12:42:10.020 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-19T12:42:10.035 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-19T12:42:10.043 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../32-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-19T12:42:10.043 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-19T12:42:10.058 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-19T12:42:10.066 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../33-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-19T12:42:10.067 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-19T12:42:10.090 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-natsort. 2026-02-19T12:42:10.098 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../34-python3-natsort_8.0.2-1_all.deb ... 2026-02-19T12:42:10.098 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-19T12:42:10.114 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-19T12:42:10.122 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../35-ceph-mgr-modules-core_20.2.0-720-g2fc421c8-1jammy_all.deb ... 2026-02-19T12:42:10.123 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking ceph-mgr-modules-core (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:10.151 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-19T12:42:10.159 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../36-libsqlite3-mod-ceph_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:10.160 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libsqlite3-mod-ceph (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:10.175 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package ceph-mgr. 2026-02-19T12:42:10.183 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../37-ceph-mgr_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:10.183 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking ceph-mgr (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:10.204 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package ceph-mon. 2026-02-19T12:42:10.212 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../38-ceph-mon_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:10.213 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking ceph-mon (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:10.284 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-19T12:42:10.292 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../39-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-19T12:42:10.293 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-19T12:42:10.309 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package ceph-osd. 2026-02-19T12:42:10.317 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../40-ceph-osd_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:10.317 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking ceph-osd (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:10.468 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package ceph. 2026-02-19T12:42:10.476 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../41-ceph_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:10.476 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking ceph (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:10.490 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package ceph-fuse. 2026-02-19T12:42:10.499 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../42-ceph-fuse_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:10.499 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking ceph-fuse (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:10.521 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package ceph-mds. 2026-02-19T12:42:10.529 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../43-ceph-mds_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:10.529 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking ceph-mds (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:10.560 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package cephadm. 2026-02-19T12:42:10.568 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../44-cephadm_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:10.569 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking cephadm (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:10.585 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-19T12:42:10.594 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../45-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-19T12:42:10.594 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-19T12:42:10.615 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-19T12:42:10.624 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../46-ceph-mgr-cephadm_20.2.0-720-g2fc421c8-1jammy_all.deb ... 2026-02-19T12:42:10.624 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking ceph-mgr-cephadm (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:10.644 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-19T12:42:10.652 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../47-python3-repoze.lru_0.7-2_all.deb ... 2026-02-19T12:42:10.653 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-19T12:42:10.668 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-routes. 2026-02-19T12:42:10.676 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../48-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-19T12:42:10.676 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-19T12:42:10.696 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-19T12:42:10.704 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../49-ceph-mgr-dashboard_20.2.0-720-g2fc421c8-1jammy_all.deb ... 2026-02-19T12:42:10.705 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking ceph-mgr-dashboard (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:11.120 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-19T12:42:11.129 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../50-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-19T12:42:11.129 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-19T12:42:11.168 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-joblib. 2026-02-19T12:42:11.176 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../51-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-19T12:42:11.177 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-19T12:42:11.205 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-19T12:42:11.213 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../52-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-19T12:42:11.214 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-19T12:42:11.228 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-sklearn. 2026-02-19T12:42:11.236 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../53-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-19T12:42:11.237 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-19T12:42:11.318 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-19T12:42:11.326 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../54-ceph-mgr-diskprediction-local_20.2.0-720-g2fc421c8-1jammy_all.deb ... 2026-02-19T12:42:11.327 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking ceph-mgr-diskprediction-local (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:11.486 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-cachetools. 2026-02-19T12:42:11.494 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../55-python3-cachetools_5.0.0-1_all.deb ... 2026-02-19T12:42:11.495 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-19T12:42:11.509 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-rsa. 2026-02-19T12:42:11.517 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../56-python3-rsa_4.8-1_all.deb ... 2026-02-19T12:42:11.518 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-19T12:42:11.536 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-google-auth. 2026-02-19T12:42:11.544 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../57-python3-google-auth_1.5.1-3_all.deb ... 2026-02-19T12:42:11.545 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-19T12:42:11.561 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-19T12:42:11.570 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../58-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-19T12:42:11.570 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-19T12:42:11.587 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-websocket. 2026-02-19T12:42:11.595 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../59-python3-websocket_1.2.3-1_all.deb ... 2026-02-19T12:42:11.595 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-19T12:42:11.613 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-19T12:42:11.621 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../60-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-19T12:42:11.621 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-19T12:42:11.716 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-19T12:42:11.725 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../61-ceph-mgr-k8sevents_20.2.0-720-g2fc421c8-1jammy_all.deb ... 2026-02-19T12:42:11.726 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking ceph-mgr-k8sevents (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:11.741 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-19T12:42:11.749 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../62-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-19T12:42:11.749 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-19T12:42:11.765 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-19T12:42:11.774 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../63-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-19T12:42:11.774 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-19T12:42:11.790 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package jq. 2026-02-19T12:42:11.798 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../64-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-19T12:42:11.799 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-19T12:42:11.813 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package socat. 2026-02-19T12:42:11.821 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../65-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-19T12:42:11.822 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-19T12:42:11.843 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package xmlstarlet. 2026-02-19T12:42:11.851 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../66-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-19T12:42:11.852 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-19T12:42:11.887 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package ceph-test. 2026-02-19T12:42:11.896 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../67-ceph-test_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:11.896 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking ceph-test (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:12.779 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package ceph-volume. 2026-02-19T12:42:12.788 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../68-ceph-volume_20.2.0-720-g2fc421c8-1jammy_all.deb ... 2026-02-19T12:42:12.788 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking ceph-volume (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:12.812 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libcephfs-daemon. 2026-02-19T12:42:12.820 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../69-libcephfs-daemon_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:12.821 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libcephfs-daemon (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:12.835 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libcephfs-proxy2. 2026-02-19T12:42:12.844 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../70-libcephfs-proxy2_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:12.844 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libcephfs-proxy2 (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:12.859 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-19T12:42:12.867 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../71-libcephfs-dev_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:12.867 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking libcephfs-dev (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:12.884 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package nvme-cli. 2026-02-19T12:42:12.892 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../72-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-19T12:42:12.893 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-19T12:42:12.925 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-19T12:42:12.933 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../73-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-19T12:42:12.934 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-19T12:42:12.965 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-19T12:42:12.973 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../74-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-19T12:42:12.973 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-19T12:42:12.989 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-pluggy. 2026-02-19T12:42:12.997 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../75-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-19T12:42:12.997 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-19T12:42:13.014 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-psutil. 2026-02-19T12:42:13.022 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../76-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-19T12:42:13.022 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-19T12:42:13.040 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-py. 2026-02-19T12:42:13.048 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../77-python3-py_1.10.0-1_all.deb ... 2026-02-19T12:42:13.049 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-19T12:42:13.068 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-pygments. 2026-02-19T12:42:13.077 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../78-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-19T12:42:13.078 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-19T12:42:13.118 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-toml. 2026-02-19T12:42:13.127 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../79-python3-toml_0.10.2-1_all.deb ... 2026-02-19T12:42:13.127 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-19T12:42:13.143 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-pytest. 2026-02-19T12:42:13.151 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../80-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-19T12:42:13.151 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-19T12:42:13.173 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-simplejson. 2026-02-19T12:42:13.181 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../81-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-19T12:42:13.181 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-19T12:42:13.200 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package python3-webob. 2026-02-19T12:42:13.208 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../82-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-19T12:42:13.209 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-19T12:42:13.232 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-19T12:42:13.241 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../83-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-19T12:42:13.241 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-19T12:42:13.309 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package radosgw. 2026-02-19T12:42:13.318 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../84-radosgw_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:13.318 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking radosgw (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:13.526 INFO:teuthology.orchestra.run.trial144.stdout:Selecting previously unselected package rbd-fuse. 2026-02-19T12:42:13.535 INFO:teuthology.orchestra.run.trial144.stdout:Preparing to unpack .../85-rbd-fuse_20.2.0-720-g2fc421c8-1jammy_amd64.deb ... 2026-02-19T12:42:13.535 INFO:teuthology.orchestra.run.trial144.stdout:Unpacking rbd-fuse (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:13.562 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-19T12:42:13.629 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-19T12:42:13.630 INFO:teuthology.orchestra.run.trial144.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-19T12:42:13.704 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-19T12:42:13.994 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-19T12:42:14.415 INFO:teuthology.orchestra.run.trial144.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-19T12:42:14.425 INFO:teuthology.orchestra.run.trial144.stdout:Setting up cephadm (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:14.469 INFO:teuthology.orchestra.run.trial144.stdout:Adding system user cephadm....done 2026-02-19T12:42:14.479 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-19T12:42:14.551 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-19T12:42:14.552 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-19T12:42:14.621 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-19T12:42:14.697 INFO:teuthology.orchestra.run.trial144.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-19T12:42:14.698 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-19T12:42:14.793 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-19T12:42:14.918 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-19T12:42:14.993 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-19T12:42:15.067 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-ceph-argparse (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:15.132 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-19T12:42:15.133 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-19T12:42:15.134 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-19T12:42:15.135 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-19T12:42:15.136 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-19T12:42:15.259 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-19T12:42:15.335 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libcephfs-proxy2 (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:15.336 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-19T12:42:15.412 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-19T12:42:15.498 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-19T12:42:15.746 INFO:teuthology.orchestra.run.trial144.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-19T12:42:15.746 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-19T12:42:15.841 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-19T12:42:15.965 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-19T12:42:16.048 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-19T12:42:16.119 INFO:teuthology.orchestra.run.trial144.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-19T12:42:16.120 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-ceph-common (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:16.207 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-19T12:42:16.704 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-19T12:42:16.709 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-19T12:42:16.786 INFO:teuthology.orchestra.run.trial144.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-19T12:42:16.787 INFO:teuthology.orchestra.run.trial144.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-19T12:42:16.788 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-19T12:42:16.861 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-19T12:42:16.933 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-19T12:42:16.933 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-19T12:42:17.011 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-19T12:42:17.086 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-19T12:42:17.165 INFO:teuthology.orchestra.run.trial144.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-19T12:42:17.166 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-19T12:42:17.249 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-19T12:42:17.250 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-19T12:42:17.325 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-19T12:42:17.409 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-19T12:42:17.483 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-19T12:42:17.484 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-19T12:42:17.615 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-19T12:42:17.687 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-19T12:42:17.688 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-19T12:42:17.768 INFO:teuthology.orchestra.run.trial144.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-19T12:42:17.769 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-19T12:42:17.900 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-19T12:42:17.901 INFO:teuthology.orchestra.run.trial144.stdout:Setting up librados2 (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:17.901 INFO:teuthology.orchestra.run.trial144.stdout:Setting up librgw2 (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:17.902 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libsqlite3-mod-ceph (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:17.903 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-19T12:42:18.415 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libcephfs2 (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:18.416 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libradosstriper1 (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:18.417 INFO:teuthology.orchestra.run.trial144.stdout:Setting up librbd1 (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:18.418 INFO:teuthology.orchestra.run.trial144.stdout:Setting up ceph-mgr-modules-core (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:18.419 INFO:teuthology.orchestra.run.trial144.stdout:Setting up ceph-fuse (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:18.488 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-19T12:42:18.488 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-19T12:42:18.935 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libcephfs-dev (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:18.936 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-rados (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:18.937 INFO:teuthology.orchestra.run.trial144.stdout:Setting up libcephfs-daemon (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:18.938 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-rbd (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:18.939 INFO:teuthology.orchestra.run.trial144.stdout:Setting up rbd-fuse (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:18.940 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-rgw (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:18.941 INFO:teuthology.orchestra.run.trial144.stdout:Setting up python3-cephfs (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:18.941 INFO:teuthology.orchestra.run.trial144.stdout:Setting up ceph-common (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:18.970 INFO:teuthology.orchestra.run.trial144.stdout:Adding group ceph....done 2026-02-19T12:42:19.006 INFO:teuthology.orchestra.run.trial144.stdout:Adding system user ceph....done 2026-02-19T12:42:19.016 INFO:teuthology.orchestra.run.trial144.stdout:Setting system user ceph properties....done 2026-02-19T12:42:19.022 INFO:teuthology.orchestra.run.trial144.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-19T12:42:19.109 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-19T12:42:19.396 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-19T12:42:19.814 INFO:teuthology.orchestra.run.trial144.stdout:Setting up ceph-test (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:19.815 INFO:teuthology.orchestra.run.trial144.stdout:Setting up radosgw (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:20.066 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-19T12:42:20.066 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-19T12:42:20.443 INFO:teuthology.orchestra.run.trial144.stdout:Setting up ceph-base (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:20.557 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-19T12:42:20.960 INFO:teuthology.orchestra.run.trial144.stdout:Setting up ceph-mds (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:21.020 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-19T12:42:21.020 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-19T12:42:21.427 INFO:teuthology.orchestra.run.trial144.stdout:Setting up ceph-mgr (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:21.504 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-19T12:42:21.504 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-19T12:42:21.911 INFO:teuthology.orchestra.run.trial144.stdout:Setting up ceph-osd (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:21.996 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-19T12:42:21.996 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-19T12:42:22.430 INFO:teuthology.orchestra.run.trial144.stdout:Setting up ceph-mgr-k8sevents (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:22.438 INFO:teuthology.orchestra.run.trial144.stdout:Setting up ceph-mgr-diskprediction-local (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:22.448 INFO:teuthology.orchestra.run.trial144.stdout:Setting up ceph-mon (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:22.504 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-19T12:42:22.505 INFO:teuthology.orchestra.run.trial144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-19T12:42:22.915 INFO:teuthology.orchestra.run.trial144.stdout:Setting up ceph-mgr-cephadm (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:22.926 INFO:teuthology.orchestra.run.trial144.stdout:Setting up ceph (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:22.927 INFO:teuthology.orchestra.run.trial144.stdout:Setting up ceph-mgr-dashboard (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:22.938 INFO:teuthology.orchestra.run.trial144.stdout:Setting up ceph-volume (20.2.0-720-g2fc421c8-1jammy) ... 2026-02-19T12:42:23.044 INFO:teuthology.orchestra.run.trial144.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-19T12:42:23.128 INFO:teuthology.orchestra.run.trial144.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.13) ... 2026-02-19T12:42:23.746 INFO:teuthology.orchestra.run.trial144.stdout: 2026-02-19T12:42:23.746 INFO:teuthology.orchestra.run.trial144.stdout:Running kernel seems to be up-to-date. 2026-02-19T12:42:23.746 INFO:teuthology.orchestra.run.trial144.stdout: 2026-02-19T12:42:23.746 INFO:teuthology.orchestra.run.trial144.stdout:The processor microcode seems to be up-to-date. 2026-02-19T12:42:23.746 INFO:teuthology.orchestra.run.trial144.stdout: 2026-02-19T12:42:23.746 INFO:teuthology.orchestra.run.trial144.stdout:No services need to be restarted. 2026-02-19T12:42:23.746 INFO:teuthology.orchestra.run.trial144.stdout: 2026-02-19T12:42:23.746 INFO:teuthology.orchestra.run.trial144.stdout:No containers need to be restarted. 2026-02-19T12:42:23.746 INFO:teuthology.orchestra.run.trial144.stdout: 2026-02-19T12:42:23.746 INFO:teuthology.orchestra.run.trial144.stdout:No user sessions are running outdated binaries. 2026-02-19T12:42:23.746 INFO:teuthology.orchestra.run.trial144.stdout: 2026-02-19T12:42:23.746 INFO:teuthology.orchestra.run.trial144.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-19T12:42:24.558 INFO:teuthology.orchestra.run.trial144.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-19T12:42:24.564 DEBUG:teuthology.parallel:result is None 2026-02-19T12:42:24.564 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2fc421c83f0b86ea21b9e4289cad0f7957e09eb1 2026-02-19T12:42:24.737 DEBUG:teuthology.orchestra.run.trial144:> dpkg-query -W -f '${Version}' ceph 2026-02-19T12:42:24.747 INFO:teuthology.orchestra.run.trial144.stdout:20.2.0-720-g2fc421c8-1jammy 2026-02-19T12:42:24.748 INFO:teuthology.packaging:The installed version of ceph is 20.2.0-720-g2fc421c8-1jammy 2026-02-19T12:42:24.748 INFO:teuthology.task.install:The correct ceph version 20.2.0-720-g2fc421c8-1jammy is installed. 2026-02-19T12:42:24.750 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-19T12:42:24.750 DEBUG:teuthology.orchestra.run.trial144:> set -ex 2026-02-19T12:42:24.750 DEBUG:teuthology.orchestra.run.trial144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-19T12:42:24.801 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-19T12:42:24.802 DEBUG:teuthology.orchestra.run.trial144:> set -ex 2026-02-19T12:42:24.802 DEBUG:teuthology.orchestra.run.trial144:> sudo dd of=/usr/bin/daemon-helper 2026-02-19T12:42:24.854 DEBUG:teuthology.orchestra.run.trial144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-19T12:42:24.908 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-19T12:42:24.909 DEBUG:teuthology.orchestra.run.trial144:> set -ex 2026-02-19T12:42:24.909 DEBUG:teuthology.orchestra.run.trial144:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-19T12:42:24.957 DEBUG:teuthology.orchestra.run.trial144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-19T12:42:25.009 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-19T12:42:25.009 DEBUG:teuthology.orchestra.run.trial144:> set -ex 2026-02-19T12:42:25.009 DEBUG:teuthology.orchestra.run.trial144:> sudo dd of=/usr/bin/stdin-killer 2026-02-19T12:42:25.058 DEBUG:teuthology.orchestra.run.trial144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-19T12:42:25.108 INFO:teuthology.run_tasks:Running task workunit... 2026-02-19T12:42:25.133 INFO:tasks.workunit:Pulling workunits from ref 624b9a7bcb2064ce17206482df6a1da007ff7675 2026-02-19T12:42:25.133 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-19T12:42:25.133 INFO:tasks.workunit:timeout=3h 2026-02-19T12:42:25.133 INFO:tasks.workunit:cleanup=True 2026-02-19T12:42:25.134 DEBUG:teuthology.orchestra.run.trial144:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-19T12:42:25.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T12:42:25.153 INFO:teuthology.orchestra.run.trial144.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-19T12:42:25.153 DEBUG:teuthology.orchestra.run.trial144:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-19T12:42:25.196 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-19T12:42:25.197 DEBUG:teuthology.orchestra.run.trial144:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-19T12:42:25.241 DEBUG:teuthology.orchestra.run.trial144:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/NitzanMordhai/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 624b9a7bcb2064ce17206482df6a1da007ff7675 2026-02-19T12:42:25.291 INFO:tasks.workunit.client.0.trial144.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-19T12:42:49.672 INFO:tasks.workunit.client.0.trial144.stderr:Note: switching to '624b9a7bcb2064ce17206482df6a1da007ff7675'. 2026-02-19T12:42:49.672 INFO:tasks.workunit.client.0.trial144.stderr: 2026-02-19T12:42:49.672 INFO:tasks.workunit.client.0.trial144.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-19T12:42:49.673 INFO:tasks.workunit.client.0.trial144.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-19T12:42:49.673 INFO:tasks.workunit.client.0.trial144.stderr:state without impacting any branches by switching back to a branch. 2026-02-19T12:42:49.673 INFO:tasks.workunit.client.0.trial144.stderr: 2026-02-19T12:42:49.673 INFO:tasks.workunit.client.0.trial144.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-19T12:42:49.673 INFO:tasks.workunit.client.0.trial144.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-19T12:42:49.673 INFO:tasks.workunit.client.0.trial144.stderr: 2026-02-19T12:42:49.673 INFO:tasks.workunit.client.0.trial144.stderr: git switch -c 2026-02-19T12:42:49.673 INFO:tasks.workunit.client.0.trial144.stderr: 2026-02-19T12:42:49.673 INFO:tasks.workunit.client.0.trial144.stderr:Or undo this operation with: 2026-02-19T12:42:49.673 INFO:tasks.workunit.client.0.trial144.stderr: 2026-02-19T12:42:49.673 INFO:tasks.workunit.client.0.trial144.stderr: git switch - 2026-02-19T12:42:49.674 INFO:tasks.workunit.client.0.trial144.stderr: 2026-02-19T12:42:49.674 INFO:tasks.workunit.client.0.trial144.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-19T12:42:49.674 INFO:tasks.workunit.client.0.trial144.stderr: 2026-02-19T12:42:49.674 INFO:tasks.workunit.client.0.trial144.stderr:HEAD is now at 624b9a7bcb2 Integration branch metadata 2026-02-19T12:42:49.680 DEBUG:teuthology.orchestra.run.trial144:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-19T12:42:49.727 DEBUG:teuthology.orchestra.run.trial144:> set -ex 2026-02-19T12:42:49.727 DEBUG:teuthology.orchestra.run.trial144:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-19T12:42:49.773 INFO:tasks.workunit:Running workunits matching scrub on client.0... 2026-02-19T12:42:49.774 INFO:tasks.workunit:Running workunit scrub/osd-mapper.sh... 2026-02-19T12:42:49.775 DEBUG:teuthology.orchestra.run.trial144:workunit test scrub/osd-mapper.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=624b9a7bcb2064ce17206482df6a1da007ff7675 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh 2026-02-19T12:42:49.827 INFO:tasks.workunit.client.0.trial144.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-19T12:42:49.833 INFO:tasks.workunit.client.0.trial144.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-19T12:42:49.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-19T12:42:49.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-19T12:42:49.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-19T12:42:49.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-19T12:42:49.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-19T12:42:49.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-19T12:42:49.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-19T12:42:49.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-19T12:42:49.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/osd-mapper 2026-02-19T12:42:49.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:11: run: local dir=td/osd-mapper 2026-02-19T12:42:49.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:12: run: shift 2026-02-19T12:42:49.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:14: run: export CEPH_MON=127.0.0.1:7144 2026-02-19T12:42:49.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:14: run: CEPH_MON=127.0.0.1:7144 2026-02-19T12:42:49.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:15: run: export CEPH_ARGS 2026-02-19T12:42:49.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:16: run: uuidgen 2026-02-19T12:42:49.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:16: run: CEPH_ARGS+='--fsid=e8d2c67c-d0fb-410d-b2e1-8374392917aa --auth-supported=none ' 2026-02-19T12:42:49.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:17: run: CEPH_ARGS+='--mon-host=127.0.0.1:7144 ' 2026-02-19T12:42:49.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:19: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-19T12:42:49.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:20: run: set 2026-02-19T12:42:49.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:20: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-19T12:42:49.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:20: run: local funcs=TEST_truncated_sna_record 2026-02-19T12:42:49.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:21: run: for func in $funcs 2026-02-19T12:42:49.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:22: run: setup td/osd-mapper 2026-02-19T12:42:49.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-mapper 2026-02-19T12:42:49.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-mapper 2026-02-19T12:42:49.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-mapper 2026-02-19T12:42:49.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:42:49.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-mapper KILL 2026-02-19T12:42:49.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:42:49.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:42:49.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:42:49.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:42:49.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:42:49.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:42:49.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:42:49.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:42:49.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:42:49.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:42:49.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:42:49.842 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:42:49.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:42:49.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:42:49.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:42:49.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:42:49.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:42:49.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:42:49.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-mapper 2026-02-19T12:42:49.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:42:49.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:42:49.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.8132 2026-02-19T12:42:49.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.8132 2026-02-19T12:42:49.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:42:49.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:42:49.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-mapper 2026-02-19T12:42:49.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:42:49.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:42:49.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.8132 2026-02-19T12:42:49.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.8132 2026-02-19T12:42:49.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:42:49.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-19T12:42:49.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-19T12:42:49.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:42:49.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-mapper 1' TERM HUP INT 2026-02-19T12:42:49.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:23: run: TEST_truncated_sna_record td/osd-mapper 2026-02-19T12:42:49.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:45: TEST_truncated_sna_record: local dir=td/osd-mapper 2026-02-19T12:42:49.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:46: TEST_truncated_sna_record: cluster_conf=(['osds_num']='3' ['pgs_in_pool']='4' ['pool_name']='test') 2026-02-19T12:42:49.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:46: TEST_truncated_sna_record: local -A cluster_conf 2026-02-19T12:42:49.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:52: TEST_truncated_sna_record: local extr_dbg=3 2026-02-19T12:42:49.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:53: TEST_truncated_sna_record: (( extr_dbg > 1 )) 2026-02-19T12:42:49.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:53: TEST_truncated_sna_record: echo 'Dir: td/osd-mapper' 2026-02-19T12:42:49.855 INFO:tasks.workunit.client.0.trial144.stdout:Dir: td/osd-mapper 2026-02-19T12:42:49.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:54: TEST_truncated_sna_record: standard_scrub_cluster td/osd-mapper cluster_conf 2026-02-19T12:42:49.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:229: standard_scrub_cluster: local dir=td/osd-mapper 2026-02-19T12:42:49.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:230: standard_scrub_cluster: local -n args=cluster_conf 2026-02-19T12:42:49.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:232: standard_scrub_cluster: local OSDS=3 2026-02-19T12:42:49.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:233: standard_scrub_cluster: local pg_num=4 2026-02-19T12:42:49.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:235: standard_scrub_cluster: local poolname=test 2026-02-19T12:42:49.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:236: standard_scrub_cluster: args['pool_name']=test 2026-02-19T12:42:49.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:238: standard_scrub_cluster: local pool_default_size=3 2026-02-19T12:42:49.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:239: standard_scrub_cluster: args['pool_default_size']=3 2026-02-19T12:42:49.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:241: standard_scrub_cluster: local extra_pars= 2026-02-19T12:42:49.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:242: standard_scrub_cluster: local debug_msg=dbg 2026-02-19T12:42:49.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:245: standard_scrub_cluster: local saved_echo_flag=x 2026-02-19T12:42:49.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:246: standard_scrub_cluster: set +x 2026-02-19T12:42:50.158 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 a423c3ab-54de-4214-98ac-a644d707e7f6 2026-02-19T12:42:50.255 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:42:50.282 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:50.280+0000 7f227b06fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:50.284 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:50.280+0000 7f227b06fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:50.285 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:50.284+0000 7f227b06fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:50.286 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:50.284+0000 7f227b06fa40 -1 bdev(0x56300a7fac00 td/osd-mapper/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:42:50.286 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:50.284+0000 7f227b06fa40 -1 bluestore(td/osd-mapper/0) _read_fsid unparsable uuid 2026-02-19T12:42:51.092 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:42:51.282 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:42:51.312 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:51.308+0000 7fba38d94a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:51.314 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:51.312+0000 7fba38d94a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:51.315 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:51.312+0000 7fba38d94a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:51.440 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:42:51.675 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:51.672+0000 7fba38d94a40 -1 Falling back to public interface 2026-02-19T12:42:52.231 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:52.228+0000 7fba38d94a40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:42:52.601 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:42:53.756 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:42:53.912 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/732514003,v1:127.0.0.1:6803/732514003] [v2:127.0.0.1:6804/732514003,v1:127.0.0.1:6805/732514003] exists,up a423c3ab-54de-4214-98ac-a644d707e7f6 2026-02-19T12:42:53.916 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 444b5105-b47a-4bb2-93ac-f2857cdde544 2026-02-19T12:42:54.078 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:42:54.107 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:54.104+0000 7f621a8f3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:54.108 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:54.104+0000 7f621a8f3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:54.109 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:54.104+0000 7f621a8f3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:54.110 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:54.108+0000 7f621a8f3a40 -1 bdev(0x55e987414c00 td/osd-mapper/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:42:54.110 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:54.108+0000 7f621a8f3a40 -1 bluestore(td/osd-mapper/1) _read_fsid unparsable uuid 2026-02-19T12:42:54.766 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:42:54.964 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:42:54.994 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:54.992+0000 7ff1bbbaea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:54.996 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:54.992+0000 7ff1bbbaea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:54.997 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:54.992+0000 7ff1bbbaea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:55.122 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:42:55.419 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:55.416+0000 7ff1bbbaea40 -1 Falling back to public interface 2026-02-19T12:42:56.050 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:56.048+0000 7ff1bbbaea40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:42:56.281 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:42:57.438 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:42:57.592 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1405302453,v1:127.0.0.1:6811/1405302453] [v2:127.0.0.1:6812/1405302453,v1:127.0.0.1:6813/1405302453] exists,up 444b5105-b47a-4bb2-93ac-f2857cdde544 2026-02-19T12:42:57.598 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 4fef2bbb-13d2-4c7e-a671-3d17e90dd637 2026-02-19T12:42:57.763 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:42:57.793 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:57.788+0000 7f50ed626a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:57.794 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:57.792+0000 7f50ed626a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:57.795 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:57.792+0000 7f50ed626a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:57.796 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:57.792+0000 7f50ed626a40 -1 bdev(0x55d823516c00 td/osd-mapper/2/block) open stat got: (1) Operation not permitted 2026-02-19T12:42:57.796 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:57.792+0000 7f50ed626a40 -1 bluestore(td/osd-mapper/2) _read_fsid unparsable uuid 2026-02-19T12:42:58.413 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T12:42:58.615 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T12:42:58.644 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:58.640+0000 7fd529029a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:58.646 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:58.644+0000 7fd529029a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:58.647 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:58.644+0000 7fd529029a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:42:58.767 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:42:59.027 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:59.024+0000 7fd529029a40 -1 Falling back to public interface 2026-02-19T12:42:59.631 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:42:59.628+0000 7fd529029a40 -1 osd.2 0 log_to_monitors true 2026-02-19T12:42:59.929 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:43:01.088 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:43:01.245 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2790805629,v1:127.0.0.1:6819/2790805629] [v2:127.0.0.1:6820/2790805629,v1:127.0.0.1:6821/2790805629] exists,up 4fef2bbb-13d2-4c7e-a671-3d17e90dd637 2026-02-19T12:43:01.409 INFO:tasks.workunit.client.0.trial144.stderr:pool 'test' created 2026-02-19T12:43:02.886 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836491 2026-02-19T12:43:04.213 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705671 2026-02-19T12:43:04.375 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574852 2026-02-19T12:43:06.229 INFO:tasks.workunit.client.0.trial144.stdout:standard_scrub_cluster: dbg: test pool is test 1 2026-02-19T12:43:06.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:55: TEST_truncated_sna_record: ceph tell 'osd.*' config set osd_stats_update_period_not_scrubbing 1 2026-02-19T12:43:06.293 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T12:43:06.293 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_stats_update_period_not_scrubbing = '' " 2026-02-19T12:43:06.293 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:43:06.300 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T12:43:06.300 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_stats_update_period_not_scrubbing = '' " 2026-02-19T12:43:06.300 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:43:06.307 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T12:43:06.307 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_stats_update_period_not_scrubbing = '' " 2026-02-19T12:43:06.307 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:43:06.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:56: TEST_truncated_sna_record: ceph tell 'osd.*' config set osd_stats_update_period_scrubbing 1 2026-02-19T12:43:06.380 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T12:43:06.381 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_stats_update_period_scrubbing = '' " 2026-02-19T12:43:06.381 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:43:06.388 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T12:43:06.388 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_stats_update_period_scrubbing = '' " 2026-02-19T12:43:06.388 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:43:06.395 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T12:43:06.395 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_stats_update_period_scrubbing = '' " 2026-02-19T12:43:06.396 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:43:06.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:58: TEST_truncated_sna_record: local osdn=3 2026-02-19T12:43:06.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:59: TEST_truncated_sna_record: local poolid=1 2026-02-19T12:43:06.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:60: TEST_truncated_sna_record: local poolname=test 2026-02-19T12:43:06.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:61: TEST_truncated_sna_record: local objname=objxxx 2026-02-19T12:43:06.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:64: TEST_truncated_sna_record: make_a_clone test objxxx snap01 snap02 2026-02-19T12:43:06.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:32: make_a_clone: local saved_echo_flag=x 2026-02-19T12:43:06.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:33: make_a_clone: set +x 2026-02-19T12:43:06.472 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap01 2026-02-19T12:43:06.576 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap02 2026-02-19T12:43:06.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:65: TEST_truncated_sna_record: make_a_clone test objxxx snap13 2026-02-19T12:43:06.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:32: make_a_clone: local saved_echo_flag=x 2026-02-19T12:43:06.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:33: make_a_clone: set +x 2026-02-19T12:43:06.679 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap13 2026-02-19T12:43:06.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:66: TEST_truncated_sna_record: make_a_clone test objxxx snap24 snap25 2026-02-19T12:43:06.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:32: make_a_clone: local saved_echo_flag=x 2026-02-19T12:43:06.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:33: make_a_clone: set +x 2026-02-19T12:43:06.779 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap24 2026-02-19T12:43:06.882 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap25 2026-02-19T12:43:06.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:67: TEST_truncated_sna_record: echo 13334 2026-02-19T12:43:06.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:67: TEST_truncated_sna_record: rados -p test put objxxx - 2026-02-19T12:43:06.907 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:70: TEST_truncated_sna_record: ceph --format=json-pretty osd map test objxxx 2026-02-19T12:43:06.907 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:70: TEST_truncated_sna_record: jq -r .pgid 2026-02-19T12:43:07.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:70: TEST_truncated_sna_record: local pgid=1.3 2026-02-19T12:43:07.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:71: TEST_truncated_sna_record: ceph --format=json-pretty osd map test objxxx 2026-02-19T12:43:07.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:71: TEST_truncated_sna_record: jq -r '.up[0]' 2026-02-19T12:43:07.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:71: TEST_truncated_sna_record: local osd=1 2026-02-19T12:43:07.223 INFO:tasks.workunit.client.0.trial144.stdout:pgid is 1.3 (primary: osd.1) 2026-02-19T12:43:07.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:72: TEST_truncated_sna_record: echo 'pgid is 1.3 (primary: osd.1)' 2026-02-19T12:43:07.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:74: TEST_truncated_sna_record: set_query_debug 1.3 2026-02-19T12:43:07.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:417: set_query_debug: local pgid=1.3 2026-02-19T12:43:07.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:419: set_query_debug: ceph pg dump pgs_brief 2026-02-19T12:43:07.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:419: set_query_debug: awk -v 'pg=^1.3' -n -e '$0 ~ pg { print(gensub(/[^0-9]*([0-9]+).*/,"\\1","g",$5)); }' 2026-02-19T12:43:07.369 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs_brief 2026-02-19T12:43:07.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:419: set_query_debug: local prim_osd=1 2026-02-19T12:43:07.381 INFO:tasks.workunit.client.0.trial144.stdout:Setting scrub debug data. Primary for 1.3 is 1 2026-02-19T12:43:07.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:421: set_query_debug: echo 'Setting scrub debug data. Primary for 1.3 is 1' 2026-02-19T12:43:07.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:422: set_query_debug: get_asok_path osd.1 2026-02-19T12:43:07.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T12:43:07.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T12:43:07.382 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:43:07.382 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:07.382 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.8132 2026-02-19T12:43:07.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.8132/ceph-osd.1.asok 2026-02-19T12:43:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:422: set_query_debug: CEPH_ARGS= 2026-02-19T12:43:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:422: set_query_debug: ceph --format=json daemon /tmp/ceph-asok.8132/ceph-osd.1.asok scrubdebug 1.3 set sessions 2026-02-19T12:43:07.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:77: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-19T12:43:07.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:77: TEST_truncated_sna_record: rados --format json-pretty -p test listsnaps objxxx 2026-02-19T12:43:07.466 INFO:tasks.workunit.client.0.trial144.stdout:{"success":true}{ 2026-02-19T12:43:07.466 INFO:tasks.workunit.client.0.trial144.stdout: "name": "objxxx", 2026-02-19T12:43:07.466 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 5, 2026-02-19T12:43:07.466 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T12:43:07.466 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:07.467 INFO:tasks.workunit.client.0.trial144.stdout: "id": 2, 2026-02-19T12:43:07.467 INFO:tasks.workunit.client.0.trial144.stdout: "snapshots": [ 2026-02-19T12:43:07.467 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:07.467 INFO:tasks.workunit.client.0.trial144.stdout: "id": 1, 2026-02-19T12:43:07.467 INFO:tasks.workunit.client.0.trial144.stdout: "name": "snap01" 2026-02-19T12:43:07.467 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T12:43:07.467 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:07.467 INFO:tasks.workunit.client.0.trial144.stdout: "id": 2, 2026-02-19T12:43:07.467 INFO:tasks.workunit.client.0.trial144.stdout: "name": "snap02" 2026-02-19T12:43:07.467 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T12:43:07.467 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T12:43:07.468 INFO:tasks.workunit.client.0.trial144.stdout: "size": 5, 2026-02-19T12:43:07.468 INFO:tasks.workunit.client.0.trial144.stdout: "overlaps": [] 2026-02-19T12:43:07.468 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T12:43:07.468 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:07.468 INFO:tasks.workunit.client.0.trial144.stdout: "id": 3, 2026-02-19T12:43:07.468 INFO:tasks.workunit.client.0.trial144.stdout: "snapshots": [ 2026-02-19T12:43:07.468 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:07.468 INFO:tasks.workunit.client.0.trial144.stdout: "id": 3, 2026-02-19T12:43:07.468 INFO:tasks.workunit.client.0.trial144.stdout: "name": "snap13" 2026-02-19T12:43:07.468 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T12:43:07.468 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T12:43:07.468 INFO:tasks.workunit.client.0.trial144.stdout: "size": 5, 2026-02-19T12:43:07.468 INFO:tasks.workunit.client.0.trial144.stdout: "overlaps": [] 2026-02-19T12:43:07.469 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T12:43:07.469 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:07.469 INFO:tasks.workunit.client.0.trial144.stdout: "id": 5, 2026-02-19T12:43:07.469 INFO:tasks.workunit.client.0.trial144.stdout: "snapshots": [ 2026-02-19T12:43:07.469 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:07.469 INFO:tasks.workunit.client.0.trial144.stdout: "id": 4, 2026-02-19T12:43:07.469 INFO:tasks.workunit.client.0.trial144.stdout: "name": "snap24" 2026-02-19T12:43:07.469 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T12:43:07.469 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:07.469 INFO:tasks.workunit.client.0.trial144.stdout: "id": 5, 2026-02-19T12:43:07.469 INFO:tasks.workunit.client.0.trial144.stdout: "name": "snap25" 2026-02-19T12:43:07.469 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T12:43:07.469 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T12:43:07.470 INFO:tasks.workunit.client.0.trial144.stdout: "size": 5, 2026-02-19T12:43:07.470 INFO:tasks.workunit.client.0.trial144.stdout: "overlaps": [] 2026-02-19T12:43:07.470 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T12:43:07.470 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:07.470 INFO:tasks.workunit.client.0.trial144.stdout: "id": "head", 2026-02-19T12:43:07.470 INFO:tasks.workunit.client.0.trial144.stdout: "snapshots": [], 2026-02-19T12:43:07.470 INFO:tasks.workunit.client.0.trial144.stdout: "size": 6 2026-02-19T12:43:07.470 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T12:43:07.470 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T12:43:07.470 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:43:07.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:80: TEST_truncated_sna_record: ceph pg 1.3 deep-scrub 2026-02-19T12:43:07.534 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T12:43:07.534 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T12:43:07.534 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T12:43:07.534 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T12:43:07.534 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:43:07.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:84: TEST_truncated_sna_record: sleep 3 2026-02-19T12:43:10.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:85: TEST_truncated_sna_record: ceph pg dump pgs 2026-02-19T12:43:10.688 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:43:10.689 INFO:tasks.workunit.client.0.trial144.stdout:1.3 4 0 0 0 0 21 0 0 7 0 7 active+clean 2026-02-19T12:43:08.129255+0000 21'7 21:41 [1,2,0] 1 [1,2,0] 1 21'7 2026-02-19T12:43:08.129189+0000 21'7 2026-02-19T12:43:08.129189+0000 0 1 periodic scrub scheduled @ 2026-02-20T12:43:08.129189+0000 4 0 2026-02-19T12:43:10.689 INFO:tasks.workunit.client.0.trial144.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:01.514183+0000 0'0 21:26 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-19T12:43:01.405630+0000 0'0 2026-02-19T12:43:01.405630+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:01.405630+0000 0 0 2026-02-19T12:43:10.689 INFO:tasks.workunit.client.0.trial144.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:01.514273+0000 0'0 21:29 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-19T12:43:01.405630+0000 0'0 2026-02-19T12:43:01.405630+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:01.405630+0000 0 0 2026-02-19T12:43:10.689 INFO:tasks.workunit.client.0.trial144.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:01.514399+0000 0'0 21:29 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T12:43:01.405630+0000 0'0 2026-02-19T12:43:01.405630+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:01.405630+0000 0 0 2026-02-19T12:43:10.689 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:10.689 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:43:10.689 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:43:10.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:86: TEST_truncated_sna_record: grep -a -q -- 'event: --^^^^---- ScrubFinished' td/osd-mapper/osd.1.log 2026-02-19T12:43:10.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:90: TEST_truncated_sna_record: ceph pg dump pgs 2026-02-19T12:43:10.841 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:43:10.841 INFO:tasks.workunit.client.0.trial144.stdout:1.3 4 0 0 0 0 21 0 0 7 0 7 active+clean 2026-02-19T12:43:08.129255+0000 21'7 21:41 [1,2,0] 1 [1,2,0] 1 21'7 2026-02-19T12:43:08.129189+0000 21'7 2026-02-19T12:43:08.129189+0000 0 1 periodic scrub scheduled @ 2026-02-20T12:43:08.129189+0000 4 0 2026-02-19T12:43:10.841 INFO:tasks.workunit.client.0.trial144.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:01.514183+0000 0'0 21:26 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-19T12:43:01.405630+0000 0'0 2026-02-19T12:43:01.405630+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:01.405630+0000 0 0 2026-02-19T12:43:10.841 INFO:tasks.workunit.client.0.trial144.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:01.514273+0000 0'0 21:29 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-19T12:43:01.405630+0000 0'0 2026-02-19T12:43:01.405630+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:01.405630+0000 0 0 2026-02-19T12:43:10.841 INFO:tasks.workunit.client.0.trial144.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:01.514399+0000 0'0 21:29 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T12:43:01.405630+0000 0'0 2026-02-19T12:43:01.405630+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:01.405630+0000 0 0 2026-02-19T12:43:10.841 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:10.841 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:43:10.842 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:43:10.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:91: TEST_truncated_sna_record: ceph osd set noscrub 2026-02-19T12:43:11.062 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T12:43:11.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:92: TEST_truncated_sna_record: ceph osd set nodeep-scrub 2026-02-19T12:43:11.265 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T12:43:11.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:93: TEST_truncated_sna_record: sleep 5 2026-02-19T12:43:16.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:94: TEST_truncated_sna_record: grep -a -q -v ERR td/osd-mapper/osd.1.log 2026-02-19T12:43:16.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:97: TEST_truncated_sna_record: kill_daemons td/osd-mapper TERM osd 2026-02-19T12:43:16.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:43:16.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:43:16.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:43:16.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:43:16.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:43:16.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:43:16.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:99: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-19T12:43:16.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:99: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 dump p 2026-02-19T12:43:16.605 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.599+0000 7f5cea783d40 1 bdev(0x55b54c0a3800 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-19T12:43:16.605 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.599+0000 7f5cea783d40 1 bdev(0x55b54c0a3800 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:16.605 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.603+0000 7f5cea783d40 1 bdev(0x55b54c0a3800 td/osd-mapper/0/block) close 2026-02-19T12:43:16.661 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.659+0000 7f5cea783d40 1 bdev(0x55b54c0a3800 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-19T12:43:16.661 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.659+0000 7f5cea783d40 1 bdev(0x55b54c0a3800 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:16.662 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.659+0000 7f5cea783d40 1 bluestore(td/osd-mapper/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-19T12:43:16.662 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.659+0000 7f5cea783d40 1 bdev(0x55b54c0a3c00 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-19T12:43:16.662 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.659+0000 7f5cea783d40 1 bdev(0x55b54c0a3c00 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:16.662 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.659+0000 7f5cea783d40 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2026-02-19T12:43:16.663 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.659+0000 7f5cea783d40 1 bluefs mount 2026-02-19T12:43:16.663 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.659+0000 7f5cea783d40 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-19T12:43:16.665 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.663+0000 7f5cea783d40 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.665 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.663+0000 7f5cea783d40 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.665 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.663+0000 7f5cea783d40 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.665 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.663+0000 7f5cea783d40 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.666 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.663+0000 7f5cea783d40 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.666 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.663+0000 7f5cea783d40 1 bluefs mount shared_bdev_used = 0 2026-02-19T12:43:16.666 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.663+0000 7f5cea783d40 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-19T12:43:16.677 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.675+0000 7f5cea783d40 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-19T12:43:16.677 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.675+0000 7f5cea783d40 1 bluestore(td/osd-mapper/0) _open_super_meta old nid_max 2051 2026-02-19T12:43:16.678 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.675+0000 7f5cea783d40 1 bluestore(td/osd-mapper/0) _open_super_meta old blobid_max 20480 2026-02-19T12:43:16.678 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.675+0000 7f5cea783d40 1 bluestore(td/osd-mapper/0) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-19T12:43:16.678 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.675+0000 7f5cea783d40 1 bluestore(td/osd-mapper/0) _open_super_meta min_alloc_size 0x1000 2026-02-19T12:43:16.678 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.675+0000 7f5cea783d40 1 freelist init 2026-02-19T12:43:16.678 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.675+0000 7f5cea783d40 1 freelist _read_cfg 2026-02-19T12:43:16.678 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.675+0000 7f5cea783d40 1 bluestore(td/osd-mapper/0) _open_fm effective freelist_type = null, freelist_alloc_size = 0x1000, min_alloc_size = 0x1000 2026-02-19T12:43:16.679 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.675+0000 7f5cea783d40 1 bluestore(td/osd-mapper/0) _init_alloc loaded 100 GiB in 3 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffc8000, fragmentation 7.62941e-08 2026-02-19T12:43:16.680 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.675+0000 7f5cea783d40 1 bluefs umount 2026-02-19T12:43:16.680 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.675+0000 7f5cea783d40 1 bdev(0x55b54c0a3c00 td/osd-mapper/0/block) close 2026-02-19T12:43:16.721 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.719+0000 7f5cea783d40 1 bdev(0x55b54c0a3c00 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-19T12:43:16.721 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.719+0000 7f5cea783d40 1 bdev(0x55b54c0a3c00 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:16.722 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.719+0000 7f5cea783d40 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2026-02-19T12:43:16.722 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.719+0000 7f5cea783d40 1 bluefs mount 2026-02-19T12:43:16.722 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.719+0000 7f5cea783d40 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-19T12:43:16.723 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.719+0000 7f5cea783d40 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.723 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.719+0000 7f5cea783d40 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.723 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.719+0000 7f5cea783d40 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.723 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.719+0000 7f5cea783d40 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.724 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.719+0000 7f5cea783d40 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.724 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.719+0000 7f5cea783d40 1 bluefs mount shared_bdev_used = 27656192 2026-02-19T12:43:16.724 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.719+0000 7f5cea783d40 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-19T12:43:16.734 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.731+0000 7f5cea783d40 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-19T12:43:16.735 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2026-02-19T12:43:16.735 INFO:tasks.workunit.client.0.trial144.stdout:00000000 0b 05 58 02 00 00 e8 d2 c6 7c d0 fb 41 0d b2 e1 |..X......|..A...| 2026-02-19T12:43:16.735 INFO:tasks.workunit.client.0.trial144.stdout:00000010 83 74 39 29 17 aa 00 00 00 00 18 00 00 00 00 00 |.t9)............| 2026-02-19T12:43:16.735 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2026-02-19T12:43:16.736 INFO:tasks.workunit.client.0.trial144.stdout:* 2026-02-19T12:43:16.736 INFO:tasks.workunit.client.0.trial144.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2026-02-19T12:43:16.736 INFO:tasks.workunit.client.0.trial144.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2026-02-19T12:43:16.736 INFO:tasks.workunit.client.0.trial144.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2026-02-19T12:43:16.736 INFO:tasks.workunit.client.0.trial144.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2026-02-19T12:43:16.736 INFO:tasks.workunit.client.0.trial144.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2026-02-19T12:43:16.736 INFO:tasks.workunit.client.0.trial144.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2026-02-19T12:43:16.736 INFO:tasks.workunit.client.0.trial144.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2026-02-19T12:43:16.736 INFO:tasks.workunit.client.0.trial144.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2026-02-19T12:43:16.736 INFO:tasks.workunit.client.0.trial144.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2026-02-19T12:43:16.736 INFO:tasks.workunit.client.0.trial144.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2026-02-19T12:43:16.736 INFO:tasks.workunit.client.0.trial144.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2026-02-19T12:43:16.737 INFO:tasks.workunit.client.0.trial144.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2026-02-19T12:43:16.737 INFO:tasks.workunit.client.0.trial144.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2026-02-19T12:43:16.737 INFO:tasks.workunit.client.0.trial144.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2026-02-19T12:43:16.737 INFO:tasks.workunit.client.0.trial144.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2026-02-19T12:43:16.737 INFO:tasks.workunit.client.0.trial144.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2026-02-19T12:43:16.737 INFO:tasks.workunit.client.0.trial144.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2026-02-19T12:43:16.737 INFO:tasks.workunit.client.0.trial144.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2026-02-19T12:43:16.737 INFO:tasks.workunit.client.0.trial144.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2026-02-19T12:43:16.737 INFO:tasks.workunit.client.0.trial144.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2026-02-19T12:43:16.737 INFO:tasks.workunit.client.0.trial144.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2026-02-19T12:43:16.737 INFO:tasks.workunit.client.0.trial144.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2026-02-19T12:43:16.737 INFO:tasks.workunit.client.0.trial144.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2026-02-19T12:43:16.737 INFO:tasks.workunit.client.0.trial144.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2026-02-19T12:43:16.738 INFO:tasks.workunit.client.0.trial144.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2026-02-19T12:43:16.738 INFO:tasks.workunit.client.0.trial144.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2026-02-19T12:43:16.738 INFO:tasks.workunit.client.0.trial144.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2026-02-19T12:43:16.738 INFO:tasks.workunit.client.0.trial144.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2026-02-19T12:43:16.738 INFO:tasks.workunit.client.0.trial144.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2026-02-19T12:43:16.738 INFO:tasks.workunit.client.0.trial144.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2026-02-19T12:43:16.738 INFO:tasks.workunit.client.0.trial144.stdout:00000220 72 65 18 00 00 00 05 00 00 00 a4 23 c3 ab 54 de |re.........#..T.| 2026-02-19T12:43:16.738 INFO:tasks.workunit.client.0.trial144.stdout:00000230 42 14 98 ac a6 44 d7 07 e7 f6 00 00 00 00 00 00 |B....D..........| 2026-02-19T12:43:16.738 INFO:tasks.workunit.client.0.trial144.stdout:00000240 00 00 18 00 00 00 4d 05 97 69 f5 f6 16 0c 01 00 |......M..i......| 2026-02-19T12:43:16.738 INFO:tasks.workunit.client.0.trial144.stdout:00000250 00 00 01 00 00 00 01 00 00 00 18 00 00 00 |..............| 2026-02-19T12:43:16.738 INFO:tasks.workunit.client.0.trial144.stdout:0000025e 2026-02-19T12:43:16.738 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:16.739 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2026-02-19T12:43:16.739 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:16.739 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:16.739 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-19T12:43:16.739 INFO:tasks.workunit.client.0.trial144.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2026-02-19T12:43:16.739 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-19T12:43:16.739 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2026-02-19T12:43:16.739 INFO:tasks.workunit.client.0.trial144.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-19T12:43:16.739 INFO:tasks.workunit.client.0.trial144.stdout:00000047 2026-02-19T12:43:16.739 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:16.739 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:16.739 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2026-02-19T12:43:16.740 INFO:tasks.workunit.client.0.trial144.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2026-02-19T12:43:16.740 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-19T12:43:16.740 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2026-02-19T12:43:16.740 INFO:tasks.workunit.client.0.trial144.stdout:0000003f 2026-02-19T12:43:16.740 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:16.740 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:16.740 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-19T12:43:16.740 INFO:tasks.workunit.client.0.trial144.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2026-02-19T12:43:16.740 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-19T12:43:16.740 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2026-02-19T12:43:16.740 INFO:tasks.workunit.client.0.trial144.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-19T12:43:16.740 INFO:tasks.workunit.client.0.trial144.stdout:00000047 2026-02-19T12:43:16.740 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:16.741 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:16.741 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:16.741 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:16.741 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:16.741 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:16.741 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:16.741 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:16.741 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:16.741 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:16.741 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:16.741 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:16.741 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:16.742 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:16.742 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:16.742 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:16.742 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:16.742 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:16.742 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:16.742 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:16.742 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:16.742 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:16.742 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:16.742 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:16.742 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:16.742 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:16.743 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:16.743 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:16.743 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:16.743 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:16.743 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:16.743 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:16.743 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:16.743 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:16.743 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:16.743 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:16.743 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2026-02-19T12:43:16.743 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:16.743 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2026-02-19T12:43:16.744 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:16.744 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.731+0000 7f5cea783d40 1 bluefs umount 2026-02-19T12:43:16.744 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.731+0000 7f5cea783d40 1 bdev(0x55b54c0a3c00 td/osd-mapper/0/block) close 2026-02-19T12:43:16.777 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.775+0000 7f5cea783d40 1 freelist shutdown 2026-02-19T12:43:16.777 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.775+0000 7f5cea783d40 1 bdev(0x55b54c0a3800 td/osd-mapper/0/block) close 2026-02-19T12:43:16.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:100: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-19T12:43:16.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:100: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2026-02-19T12:43:16.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:100: TEST_truncated_sna_record: grep -a SNA_ 2026-02-19T12:43:16.829 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.827+0000 7fa5308d5d40 1 bdev(0x55757115d800 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-19T12:43:16.829 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.827+0000 7fa5308d5d40 1 bdev(0x55757115d800 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:16.830 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.827+0000 7fa5308d5d40 1 bdev(0x55757115d800 td/osd-mapper/2/block) close 2026-02-19T12:43:16.877 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.875+0000 7fa5308d5d40 1 bdev(0x55757115d800 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-19T12:43:16.878 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.875+0000 7fa5308d5d40 1 bdev(0x55757115d800 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:16.878 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.875+0000 7fa5308d5d40 1 bluestore(td/osd-mapper/2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-19T12:43:16.878 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.875+0000 7fa5308d5d40 1 bdev(0x55757115dc00 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-19T12:43:16.878 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.875+0000 7fa5308d5d40 1 bdev(0x55757115dc00 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:16.878 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.875+0000 7fa5308d5d40 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2026-02-19T12:43:16.878 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.875+0000 7fa5308d5d40 1 bluefs mount 2026-02-19T12:43:16.878 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.875+0000 7fa5308d5d40 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-19T12:43:16.881 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.879+0000 7fa5308d5d40 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.881 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.879+0000 7fa5308d5d40 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.881 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.879+0000 7fa5308d5d40 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.881 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.879+0000 7fa5308d5d40 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.882 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.879+0000 7fa5308d5d40 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.882 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.879+0000 7fa5308d5d40 1 bluefs mount shared_bdev_used = 0 2026-02-19T12:43:16.882 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.879+0000 7fa5308d5d40 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-19T12:43:16.893 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.891+0000 7fa5308d5d40 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-19T12:43:16.894 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.891+0000 7fa5308d5d40 1 bluestore(td/osd-mapper/2) _open_super_meta old nid_max 2051 2026-02-19T12:43:16.894 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.891+0000 7fa5308d5d40 1 bluestore(td/osd-mapper/2) _open_super_meta old blobid_max 20480 2026-02-19T12:43:16.894 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.891+0000 7fa5308d5d40 1 bluestore(td/osd-mapper/2) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-19T12:43:16.894 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.891+0000 7fa5308d5d40 1 bluestore(td/osd-mapper/2) _open_super_meta min_alloc_size 0x1000 2026-02-19T12:43:16.894 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.891+0000 7fa5308d5d40 1 freelist init 2026-02-19T12:43:16.894 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.891+0000 7fa5308d5d40 1 freelist _read_cfg 2026-02-19T12:43:16.895 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.891+0000 7fa5308d5d40 1 bluestore(td/osd-mapper/2) _open_fm effective freelist_type = null, freelist_alloc_size = 0x1000, min_alloc_size = 0x1000 2026-02-19T12:43:16.896 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.891+0000 7fa5308d5d40 1 bluestore(td/osd-mapper/2) _init_alloc loaded 100 GiB in 3 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffc8000, fragmentation 7.62941e-08 2026-02-19T12:43:16.896 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.891+0000 7fa5308d5d40 1 bluefs umount 2026-02-19T12:43:16.896 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.891+0000 7fa5308d5d40 1 bdev(0x55757115dc00 td/osd-mapper/2/block) close 2026-02-19T12:43:16.953 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.951+0000 7fa5308d5d40 1 bdev(0x55757115dc00 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-19T12:43:16.953 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.951+0000 7fa5308d5d40 1 bdev(0x55757115dc00 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:16.954 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.951+0000 7fa5308d5d40 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2026-02-19T12:43:16.954 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.951+0000 7fa5308d5d40 1 bluefs mount 2026-02-19T12:43:16.954 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.951+0000 7fa5308d5d40 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-19T12:43:16.955 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.951+0000 7fa5308d5d40 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.955 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.951+0000 7fa5308d5d40 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.955 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.951+0000 7fa5308d5d40 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.955 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.951+0000 7fa5308d5d40 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.955 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.951+0000 7fa5308d5d40 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:16.956 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.951+0000 7fa5308d5d40 1 bluefs mount shared_bdev_used = 27656192 2026-02-19T12:43:16.956 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.951+0000 7fa5308d5d40 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-19T12:43:16.966 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.963+0000 7fa5308d5d40 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-19T12:43:16.967 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.963+0000 7fa5308d5d40 1 bluefs umount 2026-02-19T12:43:16.967 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:16.963+0000 7fa5308d5d40 1 bdev(0x55757115dc00 td/osd-mapper/2/block) close 2026-02-19T12:43:17.025 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.023+0000 7fa5308d5d40 1 freelist shutdown 2026-02-19T12:43:17.025 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.023+0000 7fa5308d5d40 1 bdev(0x55757115d800 td/osd-mapper/2/block) close 2026-02-19T12:43:17.085 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:17.085 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:17.085 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:17.085 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:17.085 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:17.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:101: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-19T12:43:17.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:101: TEST_truncated_sna_record: grep -a SNA_ /tmp/oo2.dump 2026-02-19T12:43:17.087 INFO:tasks.workunit.client.0.trial144.stderr:grep: /tmp/oo2.dump: No such file or directory 2026-02-19T12:43:17.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:102: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-19T12:43:17.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:102: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2026-02-19T12:43:17.239 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2026-02-19T12:43:17.239 INFO:tasks.workunit.client.0.trial144.stdout:00000000 0b 05 58 02 00 00 e8 d2 c6 7c d0 fb 41 0d b2 e1 |..X......|..A...| 2026-02-19T12:43:17.239 INFO:tasks.workunit.client.0.trial144.stdout:00000010 83 74 39 29 17 aa 02 00 00 00 18 00 00 00 00 00 |.t9)............| 2026-02-19T12:43:17.239 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2026-02-19T12:43:17.240 INFO:tasks.workunit.client.0.trial144.stdout:* 2026-02-19T12:43:17.240 INFO:tasks.workunit.client.0.trial144.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2026-02-19T12:43:17.240 INFO:tasks.workunit.client.0.trial144.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2026-02-19T12:43:17.240 INFO:tasks.workunit.client.0.trial144.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2026-02-19T12:43:17.240 INFO:tasks.workunit.client.0.trial144.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2026-02-19T12:43:17.240 INFO:tasks.workunit.client.0.trial144.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2026-02-19T12:43:17.240 INFO:tasks.workunit.client.0.trial144.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2026-02-19T12:43:17.240 INFO:tasks.workunit.client.0.trial144.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2026-02-19T12:43:17.240 INFO:tasks.workunit.client.0.trial144.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2026-02-19T12:43:17.240 INFO:tasks.workunit.client.0.trial144.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2026-02-19T12:43:17.240 INFO:tasks.workunit.client.0.trial144.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2026-02-19T12:43:17.241 INFO:tasks.workunit.client.0.trial144.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2026-02-19T12:43:17.241 INFO:tasks.workunit.client.0.trial144.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2026-02-19T12:43:17.241 INFO:tasks.workunit.client.0.trial144.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2026-02-19T12:43:17.241 INFO:tasks.workunit.client.0.trial144.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2026-02-19T12:43:17.241 INFO:tasks.workunit.client.0.trial144.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2026-02-19T12:43:17.241 INFO:tasks.workunit.client.0.trial144.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2026-02-19T12:43:17.241 INFO:tasks.workunit.client.0.trial144.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2026-02-19T12:43:17.241 INFO:tasks.workunit.client.0.trial144.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2026-02-19T12:43:17.241 INFO:tasks.workunit.client.0.trial144.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2026-02-19T12:43:17.241 INFO:tasks.workunit.client.0.trial144.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2026-02-19T12:43:17.241 INFO:tasks.workunit.client.0.trial144.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2026-02-19T12:43:17.241 INFO:tasks.workunit.client.0.trial144.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2026-02-19T12:43:17.241 INFO:tasks.workunit.client.0.trial144.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2026-02-19T12:43:17.242 INFO:tasks.workunit.client.0.trial144.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2026-02-19T12:43:17.242 INFO:tasks.workunit.client.0.trial144.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2026-02-19T12:43:17.242 INFO:tasks.workunit.client.0.trial144.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2026-02-19T12:43:17.242 INFO:tasks.workunit.client.0.trial144.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2026-02-19T12:43:17.242 INFO:tasks.workunit.client.0.trial144.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2026-02-19T12:43:17.242 INFO:tasks.workunit.client.0.trial144.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2026-02-19T12:43:17.242 INFO:tasks.workunit.client.0.trial144.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2026-02-19T12:43:17.242 INFO:tasks.workunit.client.0.trial144.stdout:00000220 72 65 18 00 00 00 0d 00 00 00 4f ef 2b bb 13 d2 |re........O.+...| 2026-02-19T12:43:17.242 INFO:tasks.workunit.client.0.trial144.stdout:00000230 4c 7e a6 71 3d 17 e9 0d d6 37 00 00 00 00 00 00 |L~.q=....7......| 2026-02-19T12:43:17.242 INFO:tasks.workunit.client.0.trial144.stdout:00000240 00 00 18 00 00 00 54 05 97 69 61 8e a8 23 01 00 |......T..ia..#..| 2026-02-19T12:43:17.242 INFO:tasks.workunit.client.0.trial144.stdout:00000250 00 00 01 00 00 00 01 00 00 00 18 00 00 00 |..............| 2026-02-19T12:43:17.242 INFO:tasks.workunit.client.0.trial144.stdout:0000025e 2026-02-19T12:43:17.243 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.243 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2026-02-19T12:43:17.243 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.243 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:17.243 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-19T12:43:17.243 INFO:tasks.workunit.client.0.trial144.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2026-02-19T12:43:17.243 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-19T12:43:17.243 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2026-02-19T12:43:17.243 INFO:tasks.workunit.client.0.trial144.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-19T12:43:17.243 INFO:tasks.workunit.client.0.trial144.stdout:00000047 2026-02-19T12:43:17.243 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.244 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:17.244 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2026-02-19T12:43:17.244 INFO:tasks.workunit.client.0.trial144.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2026-02-19T12:43:17.244 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-19T12:43:17.244 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2026-02-19T12:43:17.244 INFO:tasks.workunit.client.0.trial144.stdout:0000003f 2026-02-19T12:43:17.244 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.244 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:17.244 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-19T12:43:17.244 INFO:tasks.workunit.client.0.trial144.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2026-02-19T12:43:17.244 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-19T12:43:17.244 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2026-02-19T12:43:17.245 INFO:tasks.workunit.client.0.trial144.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-19T12:43:17.245 INFO:tasks.workunit.client.0.trial144.stdout:00000047 2026-02-19T12:43:17.245 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.245 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:17.245 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:17.245 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:17.245 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:17.245 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:17.245 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:17.245 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.245 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:17.245 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:17.245 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:17.246 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:17.246 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:17.246 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:17.246 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.246 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:17.246 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:17.246 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:17.246 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:17.246 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:17.246 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:17.246 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.246 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:17.247 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:17.247 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:17.247 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:17.247 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:17.247 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:17.247 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.247 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:17.247 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:17.247 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:17.247 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:17.247 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:17.247 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:17.248 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.248 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2026-02-19T12:43:17.248 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.248 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2026-02-19T12:43:17.248 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2026-02-19T12:43:17.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_000000000000000[0-9]_000000000000000' 2026-02-19T12:43:17.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-19T12:43:17.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: wc -l 2026-02-19T12:43:17.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: local num_sna_b4=5 2026-02-19T12:43:17.542 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: expr 3 - 1 2026-02-19T12:43:17.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: seq 0 2 2026-02-19T12:43:17.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-19T12:43:17.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:108: TEST_truncated_sna_record: kvdir=td/osd-mapper/0 2026-02-19T12:43:17.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:109: TEST_truncated_sna_record: echo 'corrupting the SnapMapper DB of osd.0 (db: td/osd-mapper/0)' 2026-02-19T12:43:17.545 INFO:tasks.workunit.client.0.trial144.stdout:corrupting the SnapMapper DB of osd.0 (db: td/osd-mapper/0) 2026-02-19T12:43:17.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: (( extr_dbg >= 3 )) 2026-02-19T12:43:17.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 dump p 2026-02-19T12:43:17.558 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.555+0000 7f7a0fb44d40 1 bdev(0x563841439800 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-19T12:43:17.558 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.555+0000 7f7a0fb44d40 1 bdev(0x563841439800 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:17.559 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.555+0000 7f7a0fb44d40 1 bdev(0x563841439800 td/osd-mapper/0/block) close 2026-02-19T12:43:17.605 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.603+0000 7f7a0fb44d40 1 bdev(0x563841439800 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-19T12:43:17.606 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.603+0000 7f7a0fb44d40 1 bdev(0x563841439800 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:17.606 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.603+0000 7f7a0fb44d40 1 bluestore(td/osd-mapper/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-19T12:43:17.606 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.603+0000 7f7a0fb44d40 1 bdev(0x563841439c00 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-19T12:43:17.606 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.603+0000 7f7a0fb44d40 1 bdev(0x563841439c00 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:17.606 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.603+0000 7f7a0fb44d40 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2026-02-19T12:43:17.606 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.603+0000 7f7a0fb44d40 1 bluefs mount 2026-02-19T12:43:17.606 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.603+0000 7f7a0fb44d40 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-19T12:43:17.610 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.607+0000 7f7a0fb44d40 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:17.610 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.607+0000 7f7a0fb44d40 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:17.610 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.607+0000 7f7a0fb44d40 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:17.610 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.607+0000 7f7a0fb44d40 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:17.610 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.607+0000 7f7a0fb44d40 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:17.610 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.607+0000 7f7a0fb44d40 1 bluefs mount shared_bdev_used = 0 2026-02-19T12:43:17.610 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.607+0000 7f7a0fb44d40 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-19T12:43:17.622 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.619+0000 7f7a0fb44d40 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-19T12:43:17.623 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.619+0000 7f7a0fb44d40 1 bluestore(td/osd-mapper/0) _open_super_meta old nid_max 2051 2026-02-19T12:43:17.623 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.619+0000 7f7a0fb44d40 1 bluestore(td/osd-mapper/0) _open_super_meta old blobid_max 20480 2026-02-19T12:43:17.623 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.619+0000 7f7a0fb44d40 1 bluestore(td/osd-mapper/0) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-19T12:43:17.623 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.619+0000 7f7a0fb44d40 1 bluestore(td/osd-mapper/0) _open_super_meta min_alloc_size 0x1000 2026-02-19T12:43:17.623 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.619+0000 7f7a0fb44d40 1 freelist init 2026-02-19T12:43:17.623 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.619+0000 7f7a0fb44d40 1 freelist _read_cfg 2026-02-19T12:43:17.623 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.619+0000 7f7a0fb44d40 1 bluestore(td/osd-mapper/0) _open_fm effective freelist_type = null, freelist_alloc_size = 0x1000, min_alloc_size = 0x1000 2026-02-19T12:43:17.624 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.619+0000 7f7a0fb44d40 1 bluestore(td/osd-mapper/0) _init_alloc loaded 100 GiB in 3 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffc8000, fragmentation 7.62941e-08 2026-02-19T12:43:17.624 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.619+0000 7f7a0fb44d40 1 bluefs umount 2026-02-19T12:43:17.624 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.619+0000 7f7a0fb44d40 1 bdev(0x563841439c00 td/osd-mapper/0/block) close 2026-02-19T12:43:17.669 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.667+0000 7f7a0fb44d40 1 bdev(0x563841439c00 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-19T12:43:17.669 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.667+0000 7f7a0fb44d40 1 bdev(0x563841439c00 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:17.670 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.667+0000 7f7a0fb44d40 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2026-02-19T12:43:17.670 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.667+0000 7f7a0fb44d40 1 bluefs mount 2026-02-19T12:43:17.670 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.667+0000 7f7a0fb44d40 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-19T12:43:17.671 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.667+0000 7f7a0fb44d40 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:17.672 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.667+0000 7f7a0fb44d40 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:17.672 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.667+0000 7f7a0fb44d40 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:17.672 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.667+0000 7f7a0fb44d40 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:17.672 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.667+0000 7f7a0fb44d40 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:17.672 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.667+0000 7f7a0fb44d40 1 bluefs mount shared_bdev_used = 27656192 2026-02-19T12:43:17.672 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.667+0000 7f7a0fb44d40 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-19T12:43:17.683 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.679+0000 7f7a0fb44d40 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-19T12:43:17.683 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2026-02-19T12:43:17.684 INFO:tasks.workunit.client.0.trial144.stdout:00000000 0b 05 58 02 00 00 e8 d2 c6 7c d0 fb 41 0d b2 e1 |..X......|..A...| 2026-02-19T12:43:17.684 INFO:tasks.workunit.client.0.trial144.stdout:00000010 83 74 39 29 17 aa 00 00 00 00 18 00 00 00 00 00 |.t9)............| 2026-02-19T12:43:17.684 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2026-02-19T12:43:17.684 INFO:tasks.workunit.client.0.trial144.stdout:* 2026-02-19T12:43:17.684 INFO:tasks.workunit.client.0.trial144.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2026-02-19T12:43:17.684 INFO:tasks.workunit.client.0.trial144.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2026-02-19T12:43:17.684 INFO:tasks.workunit.client.0.trial144.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2026-02-19T12:43:17.684 INFO:tasks.workunit.client.0.trial144.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2026-02-19T12:43:17.684 INFO:tasks.workunit.client.0.trial144.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2026-02-19T12:43:17.684 INFO:tasks.workunit.client.0.trial144.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2026-02-19T12:43:17.684 INFO:tasks.workunit.client.0.trial144.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2026-02-19T12:43:17.684 INFO:tasks.workunit.client.0.trial144.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2026-02-19T12:43:17.685 INFO:tasks.workunit.client.0.trial144.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2026-02-19T12:43:17.686 INFO:tasks.workunit.client.0.trial144.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2026-02-19T12:43:17.686 INFO:tasks.workunit.client.0.trial144.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2026-02-19T12:43:17.686 INFO:tasks.workunit.client.0.trial144.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2026-02-19T12:43:17.686 INFO:tasks.workunit.client.0.trial144.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2026-02-19T12:43:17.686 INFO:tasks.workunit.client.0.trial144.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2026-02-19T12:43:17.686 INFO:tasks.workunit.client.0.trial144.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2026-02-19T12:43:17.686 INFO:tasks.workunit.client.0.trial144.stdout:00000220 72 65 18 00 00 00 05 00 00 00 a4 23 c3 ab 54 de |re.........#..T.| 2026-02-19T12:43:17.686 INFO:tasks.workunit.client.0.trial144.stdout:00000230 42 14 98 ac a6 44 d7 07 e7 f6 00 00 00 00 00 00 |B....D..........| 2026-02-19T12:43:17.686 INFO:tasks.workunit.client.0.trial144.stdout:00000240 00 00 18 00 00 00 4d 05 97 69 f5 f6 16 0c 01 00 |......M..i......| 2026-02-19T12:43:17.686 INFO:tasks.workunit.client.0.trial144.stdout:00000250 00 00 01 00 00 00 01 00 00 00 18 00 00 00 |..............| 2026-02-19T12:43:17.686 INFO:tasks.workunit.client.0.trial144.stdout:0000025e 2026-02-19T12:43:17.686 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.686 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2026-02-19T12:43:17.686 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.686 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout:00000047 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout:0000003f 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:17.687 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-19T12:43:17.688 INFO:tasks.workunit.client.0.trial144.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2026-02-19T12:43:17.688 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-19T12:43:17.688 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2026-02-19T12:43:17.688 INFO:tasks.workunit.client.0.trial144.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-19T12:43:17.688 INFO:tasks.workunit.client.0.trial144.stdout:00000047 2026-02-19T12:43:17.688 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.688 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:17.688 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:17.688 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:17.688 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:17.688 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:17.688 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:17.688 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.688 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:17.688 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:17.689 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:17.690 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:17.690 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:17.690 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.690 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:17.690 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:17.690 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:17.690 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:17.690 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:17.690 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:17.690 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.690 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2026-02-19T12:43:17.690 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.690 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2026-02-19T12:43:17.690 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:17.691 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.679+0000 7f7a0fb44d40 1 bluefs umount 2026-02-19T12:43:17.691 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.679+0000 7f7a0fb44d40 1 bdev(0x563841439c00 td/osd-mapper/0/block) close 2026-02-19T12:43:17.729 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.727+0000 7f7a0fb44d40 1 freelist shutdown 2026-02-19T12:43:17.729 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:17.727+0000 7f7a0fb44d40 1 bdev(0x563841439800 td/osd-mapper/0/block) close 2026-02-19T12:43:17.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 dump p 2026-02-19T12:43:17.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_0000000000000003_000000000000000' 2026-02-19T12:43:17.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-19T12:43:18.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: KY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:18.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-19T12:43:18.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: echo 'SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..' 2026-02-19T12:43:18.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: cat -v 2026-02-19T12:43:18.011 INFO:tasks.workunit.client.0.trial144.stdout:SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:18.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: mktemp -p /tmp --suffix=_the_val 2026-02-19T12:43:18.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: tmp_fn1=/tmp/tmp.viXK49wOli_the_val 2026-02-19T12:43:18.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-19T12:43:18.012 INFO:tasks.workunit.client.0.trial144.stdout:Value dumped in: /tmp/tmp.viXK49wOli_the_val 2026-02-19T12:43:18.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: echo 'Value dumped in: /tmp/tmp.viXK49wOli_the_val' 2026-02-19T12:43:18.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:119: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 get p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. out /tmp/tmp.viXK49wOli_the_val 2026-02-19T12:43:18.155 INFO:tasks.workunit.client.0.trial144.stdout:(p, %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..) 2026-02-19T12:43:18.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-19T12:43:18.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: od -xc /tmp/tmp.viXK49wOli_the_val 2026-02-19T12:43:18.258 INFO:tasks.workunit.client.0.trial144.stdout:0000000 0101 0035 0000 0003 0000 0000 0000 0304 2026-02-19T12:43:18.258 INFO:tasks.workunit.client.0.trial144.stdout: 001 001 5 \0 \0 \0 003 \0 \0 \0 \0 \0 \0 \0 004 003 2026-02-19T12:43:18.258 INFO:tasks.workunit.client.0.trial144.stdout:0000020 0027 0000 0000 0000 0006 0000 626f 786a 2026-02-19T12:43:18.258 INFO:tasks.workunit.client.0.trial144.stdout: ' \0 \0 \0 \0 \0 \0 \0 006 \0 \0 \0 o b j x 2026-02-19T12:43:18.258 INFO:tasks.workunit.client.0.trial144.stdout:0000040 7878 0003 0000 0000 0000 75af 85d0 0000 2026-02-19T12:43:18.258 INFO:tasks.workunit.client.0.trial144.stdout: x x 003 \0 \0 \0 \0 \0 \0 \0 257 u 320 205 \0 \0 2026-02-19T12:43:18.259 INFO:tasks.workunit.client.0.trial144.stdout:0000060 0000 0100 0000 0000 0000 0000 2026-02-19T12:43:18.259 INFO:tasks.workunit.client.0.trial144.stdout: \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 2026-02-19T12:43:18.259 INFO:tasks.workunit.client.0.trial144.stdout:0000073 2026-02-19T12:43:18.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:122: TEST_truncated_sna_record: NKY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2026-02-19T12:43:18.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:123: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 rm p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:18.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:124: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 set p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 in /tmp/tmp.viXK49wOli_the_val 2026-02-19T12:43:18.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:126: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-19T12:43:18.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-19T12:43:18.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:108: TEST_truncated_sna_record: kvdir=td/osd-mapper/1 2026-02-19T12:43:18.737 INFO:tasks.workunit.client.0.trial144.stdout:corrupting the SnapMapper DB of osd.1 (db: td/osd-mapper/1) 2026-02-19T12:43:18.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:109: TEST_truncated_sna_record: echo 'corrupting the SnapMapper DB of osd.1 (db: td/osd-mapper/1)' 2026-02-19T12:43:18.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: (( extr_dbg >= 3 )) 2026-02-19T12:43:18.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2026-02-19T12:43:18.749 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.743+0000 7f5dc593ad40 1 bdev(0x561825b59800 td/osd-mapper/1/block) open path td/osd-mapper/1/block 2026-02-19T12:43:18.749 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.743+0000 7f5dc593ad40 1 bdev(0x561825b59800 td/osd-mapper/1/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:18.749 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.747+0000 7f5dc593ad40 1 bdev(0x561825b59800 td/osd-mapper/1/block) close 2026-02-19T12:43:18.809 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.807+0000 7f5dc593ad40 1 bdev(0x561825b59800 td/osd-mapper/1/block) open path td/osd-mapper/1/block 2026-02-19T12:43:18.809 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.807+0000 7f5dc593ad40 1 bdev(0x561825b59800 td/osd-mapper/1/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:18.810 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.807+0000 7f5dc593ad40 1 bluestore(td/osd-mapper/1) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-19T12:43:18.810 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.807+0000 7f5dc593ad40 1 bdev(0x561825b59c00 td/osd-mapper/1/block) open path td/osd-mapper/1/block 2026-02-19T12:43:18.810 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.807+0000 7f5dc593ad40 1 bdev(0x561825b59c00 td/osd-mapper/1/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:18.811 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.807+0000 7f5dc593ad40 1 bluefs add_block_device bdev 1 path td/osd-mapper/1/block size 100 GiB 2026-02-19T12:43:18.811 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.807+0000 7f5dc593ad40 1 bluefs mount 2026-02-19T12:43:18.811 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.807+0000 7f5dc593ad40 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-19T12:43:18.813 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.811+0000 7f5dc593ad40 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:18.813 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.811+0000 7f5dc593ad40 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:18.813 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.811+0000 7f5dc593ad40 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:18.814 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.811+0000 7f5dc593ad40 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:18.814 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.811+0000 7f5dc593ad40 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:18.814 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.811+0000 7f5dc593ad40 1 bluefs mount shared_bdev_used = 0 2026-02-19T12:43:18.814 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.811+0000 7f5dc593ad40 1 bluestore(td/osd-mapper/1) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-19T12:43:18.826 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.823+0000 7f5dc593ad40 1 bluestore(td/osd-mapper/1) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-19T12:43:18.826 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.823+0000 7f5dc593ad40 1 bluestore(td/osd-mapper/1) _open_super_meta old nid_max 2051 2026-02-19T12:43:18.827 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.823+0000 7f5dc593ad40 1 bluestore(td/osd-mapper/1) _open_super_meta old blobid_max 20480 2026-02-19T12:43:18.827 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.823+0000 7f5dc593ad40 1 bluestore(td/osd-mapper/1) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-19T12:43:18.827 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.823+0000 7f5dc593ad40 1 bluestore(td/osd-mapper/1) _open_super_meta min_alloc_size 0x1000 2026-02-19T12:43:18.827 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.823+0000 7f5dc593ad40 1 freelist init 2026-02-19T12:43:18.827 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.823+0000 7f5dc593ad40 1 freelist _read_cfg 2026-02-19T12:43:18.827 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.823+0000 7f5dc593ad40 1 bluestore(td/osd-mapper/1) _open_fm effective freelist_type = null, freelist_alloc_size = 0x1000, min_alloc_size = 0x1000 2026-02-19T12:43:18.828 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.823+0000 7f5dc593ad40 1 bluestore(td/osd-mapper/1) _init_alloc loaded 100 GiB in 3 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffc8000, fragmentation 7.62941e-08 2026-02-19T12:43:18.828 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.823+0000 7f5dc593ad40 1 bluefs umount 2026-02-19T12:43:18.828 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.823+0000 7f5dc593ad40 1 bdev(0x561825b59c00 td/osd-mapper/1/block) close 2026-02-19T12:43:18.877 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.875+0000 7f5dc593ad40 1 bdev(0x561825b59c00 td/osd-mapper/1/block) open path td/osd-mapper/1/block 2026-02-19T12:43:18.877 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.875+0000 7f5dc593ad40 1 bdev(0x561825b59c00 td/osd-mapper/1/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:18.878 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.875+0000 7f5dc593ad40 1 bluefs add_block_device bdev 1 path td/osd-mapper/1/block size 100 GiB 2026-02-19T12:43:18.878 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.875+0000 7f5dc593ad40 1 bluefs mount 2026-02-19T12:43:18.878 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.875+0000 7f5dc593ad40 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-19T12:43:18.879 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.875+0000 7f5dc593ad40 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:18.879 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.875+0000 7f5dc593ad40 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:18.880 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.875+0000 7f5dc593ad40 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:18.880 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.875+0000 7f5dc593ad40 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:18.880 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.875+0000 7f5dc593ad40 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:18.880 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.875+0000 7f5dc593ad40 1 bluefs mount shared_bdev_used = 27656192 2026-02-19T12:43:18.880 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.875+0000 7f5dc593ad40 1 bluestore(td/osd-mapper/1) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-19T12:43:18.893 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.891+0000 7f5dc593ad40 1 bluestore(td/osd-mapper/1) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-19T12:43:18.894 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2026-02-19T12:43:18.894 INFO:tasks.workunit.client.0.trial144.stdout:00000000 0b 05 58 02 00 00 e8 d2 c6 7c d0 fb 41 0d b2 e1 |..X......|..A...| 2026-02-19T12:43:18.894 INFO:tasks.workunit.client.0.trial144.stdout:00000010 83 74 39 29 17 aa 01 00 00 00 18 00 00 00 00 00 |.t9)............| 2026-02-19T12:43:18.895 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2026-02-19T12:43:18.895 INFO:tasks.workunit.client.0.trial144.stdout:* 2026-02-19T12:43:18.895 INFO:tasks.workunit.client.0.trial144.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2026-02-19T12:43:18.895 INFO:tasks.workunit.client.0.trial144.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2026-02-19T12:43:18.895 INFO:tasks.workunit.client.0.trial144.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2026-02-19T12:43:18.895 INFO:tasks.workunit.client.0.trial144.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2026-02-19T12:43:18.895 INFO:tasks.workunit.client.0.trial144.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2026-02-19T12:43:18.895 INFO:tasks.workunit.client.0.trial144.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2026-02-19T12:43:18.895 INFO:tasks.workunit.client.0.trial144.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2026-02-19T12:43:18.895 INFO:tasks.workunit.client.0.trial144.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2026-02-19T12:43:18.895 INFO:tasks.workunit.client.0.trial144.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2026-02-19T12:43:18.895 INFO:tasks.workunit.client.0.trial144.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2026-02-19T12:43:18.896 INFO:tasks.workunit.client.0.trial144.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2026-02-19T12:43:18.896 INFO:tasks.workunit.client.0.trial144.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2026-02-19T12:43:18.896 INFO:tasks.workunit.client.0.trial144.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2026-02-19T12:43:18.896 INFO:tasks.workunit.client.0.trial144.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2026-02-19T12:43:18.896 INFO:tasks.workunit.client.0.trial144.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2026-02-19T12:43:18.896 INFO:tasks.workunit.client.0.trial144.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2026-02-19T12:43:18.896 INFO:tasks.workunit.client.0.trial144.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2026-02-19T12:43:18.896 INFO:tasks.workunit.client.0.trial144.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2026-02-19T12:43:18.896 INFO:tasks.workunit.client.0.trial144.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2026-02-19T12:43:18.896 INFO:tasks.workunit.client.0.trial144.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2026-02-19T12:43:18.896 INFO:tasks.workunit.client.0.trial144.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2026-02-19T12:43:18.896 INFO:tasks.workunit.client.0.trial144.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2026-02-19T12:43:18.896 INFO:tasks.workunit.client.0.trial144.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2026-02-19T12:43:18.897 INFO:tasks.workunit.client.0.trial144.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2026-02-19T12:43:18.897 INFO:tasks.workunit.client.0.trial144.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2026-02-19T12:43:18.897 INFO:tasks.workunit.client.0.trial144.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2026-02-19T12:43:18.897 INFO:tasks.workunit.client.0.trial144.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2026-02-19T12:43:18.897 INFO:tasks.workunit.client.0.trial144.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2026-02-19T12:43:18.897 INFO:tasks.workunit.client.0.trial144.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2026-02-19T12:43:18.897 INFO:tasks.workunit.client.0.trial144.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2026-02-19T12:43:18.897 INFO:tasks.workunit.client.0.trial144.stdout:00000220 72 65 18 00 00 00 09 00 00 00 44 4b 51 05 b4 7a |re........DKQ..z| 2026-02-19T12:43:18.897 INFO:tasks.workunit.client.0.trial144.stdout:00000230 4b b2 93 ac f2 85 7c dd e5 44 00 00 00 00 00 00 |K.....|..D......| 2026-02-19T12:43:18.897 INFO:tasks.workunit.client.0.trial144.stdout:00000240 00 00 18 00 00 00 51 05 97 69 ad 13 80 04 01 00 |......Q..i......| 2026-02-19T12:43:18.897 INFO:tasks.workunit.client.0.trial144.stdout:00000250 00 00 01 00 00 00 01 00 00 00 18 00 00 00 |..............| 2026-02-19T12:43:18.897 INFO:tasks.workunit.client.0.trial144.stdout:0000025e 2026-02-19T12:43:18.898 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:18.898 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2026-02-19T12:43:18.898 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:18.898 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:18.898 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-19T12:43:18.898 INFO:tasks.workunit.client.0.trial144.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2026-02-19T12:43:18.898 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-19T12:43:18.898 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2026-02-19T12:43:18.898 INFO:tasks.workunit.client.0.trial144.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-19T12:43:18.898 INFO:tasks.workunit.client.0.trial144.stdout:00000047 2026-02-19T12:43:18.898 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:18.899 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:18.899 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2026-02-19T12:43:18.899 INFO:tasks.workunit.client.0.trial144.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2026-02-19T12:43:18.899 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-19T12:43:18.899 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2026-02-19T12:43:18.899 INFO:tasks.workunit.client.0.trial144.stdout:0000003f 2026-02-19T12:43:18.899 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:18.899 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:18.899 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-19T12:43:18.899 INFO:tasks.workunit.client.0.trial144.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2026-02-19T12:43:18.899 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-19T12:43:18.900 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2026-02-19T12:43:18.900 INFO:tasks.workunit.client.0.trial144.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-19T12:43:18.900 INFO:tasks.workunit.client.0.trial144.stdout:00000047 2026-02-19T12:43:18.900 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:18.900 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:18.900 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:18.900 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:18.900 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:18.900 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:18.900 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:18.900 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:18.900 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:18.900 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:18.901 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:18.901 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:18.901 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:18.901 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:18.901 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:18.901 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:18.901 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:18.901 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:18.902 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:18.902 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:18.902 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:18.902 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:18.902 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:18.902 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:18.902 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:18.902 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:18.902 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:18.902 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:18.902 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:18.903 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:18.903 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:18.903 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:18.903 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:18.903 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:18.903 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:18.903 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:18.903 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2026-02-19T12:43:18.903 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:18.903 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2026-02-19T12:43:18.903 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:18.904 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.891+0000 7f5dc593ad40 1 bluefs umount 2026-02-19T12:43:18.904 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.891+0000 7f5dc593ad40 1 bdev(0x561825b59c00 td/osd-mapper/1/block) close 2026-02-19T12:43:18.937 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.935+0000 7f5dc593ad40 1 freelist shutdown 2026-02-19T12:43:18.937 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:18.935+0000 7f5dc593ad40 1 bdev(0x561825b59800 td/osd-mapper/1/block) close 2026-02-19T12:43:18.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2026-02-19T12:43:18.978 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_0000000000000003_000000000000000' 2026-02-19T12:43:18.978 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-19T12:43:19.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: KY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:19.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-19T12:43:19.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: echo 'SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..' 2026-02-19T12:43:19.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: cat -v 2026-02-19T12:43:19.214 INFO:tasks.workunit.client.0.trial144.stdout:SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:19.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: mktemp -p /tmp --suffix=_the_val 2026-02-19T12:43:19.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: tmp_fn1=/tmp/tmp.hOyq0PACWR_the_val 2026-02-19T12:43:19.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-19T12:43:19.216 INFO:tasks.workunit.client.0.trial144.stdout:Value dumped in: /tmp/tmp.hOyq0PACWR_the_val 2026-02-19T12:43:19.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: echo 'Value dumped in: /tmp/tmp.hOyq0PACWR_the_val' 2026-02-19T12:43:19.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:119: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 get p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. out /tmp/tmp.hOyq0PACWR_the_val 2026-02-19T12:43:19.346 INFO:tasks.workunit.client.0.trial144.stdout:(p, %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..) 2026-02-19T12:43:19.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-19T12:43:19.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: od -xc /tmp/tmp.hOyq0PACWR_the_val 2026-02-19T12:43:19.429 INFO:tasks.workunit.client.0.trial144.stdout:0000000 0101 0035 0000 0003 0000 0000 0000 0304 2026-02-19T12:43:19.429 INFO:tasks.workunit.client.0.trial144.stdout: 001 001 5 \0 \0 \0 003 \0 \0 \0 \0 \0 \0 \0 004 003 2026-02-19T12:43:19.430 INFO:tasks.workunit.client.0.trial144.stdout:0000020 0027 0000 0000 0000 0006 0000 626f 786a 2026-02-19T12:43:19.430 INFO:tasks.workunit.client.0.trial144.stdout: ' \0 \0 \0 \0 \0 \0 \0 006 \0 \0 \0 o b j x 2026-02-19T12:43:19.430 INFO:tasks.workunit.client.0.trial144.stdout:0000040 7878 0003 0000 0000 0000 75af 85d0 0000 2026-02-19T12:43:19.430 INFO:tasks.workunit.client.0.trial144.stdout: x x 003 \0 \0 \0 \0 \0 \0 \0 257 u 320 205 \0 \0 2026-02-19T12:43:19.430 INFO:tasks.workunit.client.0.trial144.stdout:0000060 0000 0100 0000 0000 0000 0000 2026-02-19T12:43:19.430 INFO:tasks.workunit.client.0.trial144.stdout: \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 2026-02-19T12:43:19.430 INFO:tasks.workunit.client.0.trial144.stdout:0000073 2026-02-19T12:43:19.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:122: TEST_truncated_sna_record: NKY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2026-02-19T12:43:19.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:123: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 rm p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:19.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:124: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 set p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 in /tmp/tmp.hOyq0PACWR_the_val 2026-02-19T12:43:19.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:126: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-19T12:43:19.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-19T12:43:19.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:108: TEST_truncated_sna_record: kvdir=td/osd-mapper/2 2026-02-19T12:43:19.905 INFO:tasks.workunit.client.0.trial144.stdout:corrupting the SnapMapper DB of osd.2 (db: td/osd-mapper/2) 2026-02-19T12:43:19.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:109: TEST_truncated_sna_record: echo 'corrupting the SnapMapper DB of osd.2 (db: td/osd-mapper/2)' 2026-02-19T12:43:19.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: (( extr_dbg >= 3 )) 2026-02-19T12:43:19.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2026-02-19T12:43:19.917 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.911+0000 7feeb1139d40 1 bdev(0x564723ec9800 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-19T12:43:19.917 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.911+0000 7feeb1139d40 1 bdev(0x564723ec9800 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:19.917 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.915+0000 7feeb1139d40 1 bdev(0x564723ec9800 td/osd-mapper/2/block) close 2026-02-19T12:43:19.977 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.975+0000 7feeb1139d40 1 bdev(0x564723ec9800 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-19T12:43:19.977 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.975+0000 7feeb1139d40 1 bdev(0x564723ec9800 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:19.978 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.975+0000 7feeb1139d40 1 bluestore(td/osd-mapper/2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-19T12:43:19.978 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.975+0000 7feeb1139d40 1 bdev(0x564723ec9c00 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-19T12:43:19.978 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.975+0000 7feeb1139d40 1 bdev(0x564723ec9c00 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:19.978 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.975+0000 7feeb1139d40 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2026-02-19T12:43:19.978 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.975+0000 7feeb1139d40 1 bluefs mount 2026-02-19T12:43:19.979 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.975+0000 7feeb1139d40 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-19T12:43:19.981 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.979+0000 7feeb1139d40 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:19.981 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.979+0000 7feeb1139d40 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:19.981 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.979+0000 7feeb1139d40 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:19.982 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.979+0000 7feeb1139d40 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:19.982 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.979+0000 7feeb1139d40 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:19.982 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.979+0000 7feeb1139d40 1 bluefs mount shared_bdev_used = 0 2026-02-19T12:43:19.982 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.979+0000 7feeb1139d40 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-19T12:43:19.994 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.991+0000 7feeb1139d40 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-19T12:43:19.994 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.991+0000 7feeb1139d40 1 bluestore(td/osd-mapper/2) _open_super_meta old nid_max 2051 2026-02-19T12:43:19.994 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.991+0000 7feeb1139d40 1 bluestore(td/osd-mapper/2) _open_super_meta old blobid_max 20480 2026-02-19T12:43:19.994 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.991+0000 7feeb1139d40 1 bluestore(td/osd-mapper/2) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-19T12:43:19.994 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.991+0000 7feeb1139d40 1 bluestore(td/osd-mapper/2) _open_super_meta min_alloc_size 0x1000 2026-02-19T12:43:19.994 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.991+0000 7feeb1139d40 1 freelist init 2026-02-19T12:43:19.994 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.991+0000 7feeb1139d40 1 freelist _read_cfg 2026-02-19T12:43:19.995 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.991+0000 7feeb1139d40 1 bluestore(td/osd-mapper/2) _open_fm effective freelist_type = null, freelist_alloc_size = 0x1000, min_alloc_size = 0x1000 2026-02-19T12:43:19.995 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.991+0000 7feeb1139d40 1 bluestore(td/osd-mapper/2) _init_alloc loaded 100 GiB in 3 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffc8000, fragmentation 7.62941e-08 2026-02-19T12:43:19.996 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.991+0000 7feeb1139d40 1 bluefs umount 2026-02-19T12:43:19.996 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:19.991+0000 7feeb1139d40 1 bdev(0x564723ec9c00 td/osd-mapper/2/block) close 2026-02-19T12:43:20.057 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.055+0000 7feeb1139d40 1 bdev(0x564723ec9c00 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-19T12:43:20.057 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.055+0000 7feeb1139d40 1 bdev(0x564723ec9c00 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-19T12:43:20.057 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.055+0000 7feeb1139d40 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2026-02-19T12:43:20.057 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.055+0000 7feeb1139d40 1 bluefs mount 2026-02-19T12:43:20.057 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.055+0000 7feeb1139d40 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-19T12:43:20.059 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.055+0000 7feeb1139d40 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:20.059 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.055+0000 7feeb1139d40 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:20.059 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.055+0000 7feeb1139d40 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:20.059 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.055+0000 7feeb1139d40 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:20.059 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.055+0000 7feeb1139d40 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-19T12:43:20.059 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.055+0000 7feeb1139d40 1 bluefs mount shared_bdev_used = 27656192 2026-02-19T12:43:20.059 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.055+0000 7feeb1139d40 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-19T12:43:20.070 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.067+0000 7feeb1139d40 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:00000000 0b 05 58 02 00 00 e8 d2 c6 7c d0 fb 41 0d b2 e1 |..X......|..A...| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:00000010 83 74 39 29 17 aa 02 00 00 00 18 00 00 00 00 00 |.t9)............| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:* 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2026-02-19T12:43:20.071 INFO:tasks.workunit.client.0.trial144.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:00000220 72 65 18 00 00 00 0d 00 00 00 4f ef 2b bb 13 d2 |re........O.+...| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:00000230 4c 7e a6 71 3d 17 e9 0d d6 37 00 00 00 00 00 00 |L~.q=....7......| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:00000240 00 00 18 00 00 00 54 05 97 69 61 8e a8 23 01 00 |......T..ia..#..| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:00000250 00 00 01 00 00 00 01 00 00 00 18 00 00 00 |..............| 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:0000025e 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:20.072 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000047 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:0000003f 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000047 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:20.073 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:20.074 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:20.075 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:20.075 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:20.075 INFO:tasks.workunit.client.0.trial144.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-19T12:43:20.075 INFO:tasks.workunit.client.0.trial144.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-19T12:43:20.075 INFO:tasks.workunit.client.0.trial144.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-19T12:43:20.075 INFO:tasks.workunit.client.0.trial144.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-19T12:43:20.075 INFO:tasks.workunit.client.0.trial144.stdout:0000003b 2026-02-19T12:43:20.075 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:20.075 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2026-02-19T12:43:20.075 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:20.075 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2026-02-19T12:43:20.075 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:20.075 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.067+0000 7feeb1139d40 1 bluefs umount 2026-02-19T12:43:20.075 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.067+0000 7feeb1139d40 1 bdev(0x564723ec9c00 td/osd-mapper/2/block) close 2026-02-19T12:43:20.117 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.115+0000 7feeb1139d40 1 freelist shutdown 2026-02-19T12:43:20.117 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:20.115+0000 7feeb1139d40 1 bdev(0x564723ec9800 td/osd-mapper/2/block) close 2026-02-19T12:43:20.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2026-02-19T12:43:20.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_0000000000000003_000000000000000' 2026-02-19T12:43:20.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-19T12:43:20.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: KY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:20.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-19T12:43:20.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: echo 'SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..' 2026-02-19T12:43:20.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: cat -v 2026-02-19T12:43:20.426 INFO:tasks.workunit.client.0.trial144.stdout:SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:20.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: mktemp -p /tmp --suffix=_the_val 2026-02-19T12:43:20.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: tmp_fn1=/tmp/tmp.IBy7bkIIrT_the_val 2026-02-19T12:43:20.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-19T12:43:20.428 INFO:tasks.workunit.client.0.trial144.stdout:Value dumped in: /tmp/tmp.IBy7bkIIrT_the_val 2026-02-19T12:43:20.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: echo 'Value dumped in: /tmp/tmp.IBy7bkIIrT_the_val' 2026-02-19T12:43:20.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:119: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 get p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. out /tmp/tmp.IBy7bkIIrT_the_val 2026-02-19T12:43:20.562 INFO:tasks.workunit.client.0.trial144.stdout:(p, %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..) 2026-02-19T12:43:20.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-19T12:43:20.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: od -xc /tmp/tmp.IBy7bkIIrT_the_val 2026-02-19T12:43:20.665 INFO:tasks.workunit.client.0.trial144.stdout:0000000 0101 0035 0000 0003 0000 0000 0000 0304 2026-02-19T12:43:20.665 INFO:tasks.workunit.client.0.trial144.stdout: 001 001 5 \0 \0 \0 003 \0 \0 \0 \0 \0 \0 \0 004 003 2026-02-19T12:43:20.666 INFO:tasks.workunit.client.0.trial144.stdout:0000020 0027 0000 0000 0000 0006 0000 626f 786a 2026-02-19T12:43:20.666 INFO:tasks.workunit.client.0.trial144.stdout: ' \0 \0 \0 \0 \0 \0 \0 006 \0 \0 \0 o b j x 2026-02-19T12:43:20.666 INFO:tasks.workunit.client.0.trial144.stdout:0000040 7878 0003 0000 0000 0000 75af 85d0 0000 2026-02-19T12:43:20.666 INFO:tasks.workunit.client.0.trial144.stdout: x x 003 \0 \0 \0 \0 \0 \0 \0 257 u 320 205 \0 \0 2026-02-19T12:43:20.666 INFO:tasks.workunit.client.0.trial144.stdout:0000060 0000 0100 0000 0000 0000 0000 2026-02-19T12:43:20.666 INFO:tasks.workunit.client.0.trial144.stdout: \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 2026-02-19T12:43:20.666 INFO:tasks.workunit.client.0.trial144.stdout:0000073 2026-02-19T12:43:20.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:122: TEST_truncated_sna_record: NKY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2026-02-19T12:43:20.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:123: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 rm p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:20.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:124: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 set p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 in /tmp/tmp.IBy7bkIIrT_the_val 2026-02-19T12:43:21.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:126: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-19T12:43:21.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:129: TEST_truncated_sna_record: orig_osd_args=' --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3 ' 2026-02-19T12:43:21.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:130: TEST_truncated_sna_record: echo --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3 2026-02-19T12:43:21.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:130: TEST_truncated_sna_record: orig_osd_args=' --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3' 2026-02-19T12:43:21.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:131: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-19T12:43:21.142 INFO:tasks.workunit.client.0.trial144.stdout:Copied OSD args: / --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3/ /--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3/ 2026-02-19T12:43:21.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:131: TEST_truncated_sna_record: echo 'Copied OSD args: / --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3/ /--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3/' 2026-02-19T12:43:21.143 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: expr 3 - 1 2026-02-19T12:43:21.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: seq 0 2 2026-02-19T12:43:21.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-19T12:43:21.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: CEPH_ARGS='--fsid=e8d2c67c-d0fb-410d-b2e1-8374392917aa --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3' 2026-02-19T12:43:21.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: activate_osd td/osd-mapper 0 2026-02-19T12:43:21.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-mapper 2026-02-19T12:43:21.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:43:21.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:43:21.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:43:21.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-mapper/0 2026-02-19T12:43:21.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=e8d2c67c-d0fb-410d-b2e1-8374392917aa --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3' 2026-02-19T12:43:21.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:43:21.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:43:21.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:43:21.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-mapper/0' 2026-02-19T12:43:21.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-mapper/0/journal' 2026-02-19T12:43:21.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:43:21.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:43:21.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-mapper' 2026-02-19T12:43:21.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:43:21.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:43:21.148 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:43:21.148 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:43:21.148 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:21.148 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.8132 2026-02-19T12:43:21.148 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.8132/$cluster-$name.asok' 2026-02-19T12:43:21.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.8132/$cluster-$name.asok' 2026-02-19T12:43:21.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:43:21.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-mapper/$name.log' 2026-02-19T12:43:21.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-mapper/$name.pid' 2026-02-19T12:43:21.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:43:21.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:43:21.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:43:21.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:43:21.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:43:21.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:43:21.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-mapper/0 2026-02-19T12:43:21.149 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:43:21.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:43:21.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=e8d2c67c-d0fb-410d-b2e1-8374392917aa --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-mapper/0 --osd-journal=td/osd-mapper/0/journal --chdir= --run-dir=td/osd-mapper '--admin-socket=/tmp/ceph-asok.8132/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-mapper/$name.log' '--pid-file=td/osd-mapper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:43:21.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-mapper/0/whoami 2026-02-19T12:43:21.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:43:21.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:43:21.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:43:21.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:43:21.181 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:21.175+0000 7fd5a8560a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:21.182 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:21.179+0000 7fd5a8560a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:21.183 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:21.179+0000 7fd5a8560a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:21.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:43:21.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:43:21.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:43:21.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:43:21.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:43:21.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:43:21.312 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:43:21.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:43:21.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:43:21.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:43:21.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:43:21.619 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:21.615+0000 7fd5a8560a40 -1 Falling back to public interface 2026-02-19T12:43:22.278 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:22.275+0000 7fd5a8560a40 -1 osd.0 24 log_to_monitors true 2026-02-19T12:43:22.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:43:22.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:43:22.487 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:43:22.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:43:22.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:43:22.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:43:22.656 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 27 up_thru 27 down_at 24 last_clean_interval [5,24) [v2:127.0.0.1:6802/4073741028,v1:127.0.0.1:6803/4073741028] [v2:127.0.0.1:6804/4073741028,v1:127.0.0.1:6805/4073741028] exists,up a423c3ab-54de-4214-98ac-a644d707e7f6 2026-02-19T12:43:22.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:43:22.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:43:22.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:43:22.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-19T12:43:22.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: CEPH_ARGS='--fsid=e8d2c67c-d0fb-410d-b2e1-8374392917aa --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3' 2026-02-19T12:43:22.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: activate_osd td/osd-mapper 1 2026-02-19T12:43:22.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-mapper 2026-02-19T12:43:22.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:43:22.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:43:22.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:43:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-mapper/1 2026-02-19T12:43:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=e8d2c67c-d0fb-410d-b2e1-8374392917aa --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3' 2026-02-19T12:43:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:43:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:43:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:43:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-mapper/1' 2026-02-19T12:43:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-mapper/1/journal' 2026-02-19T12:43:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:43:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:43:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-mapper' 2026-02-19T12:43:22.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:43:22.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:43:22.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:43:22.661 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:43:22.661 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:22.661 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.8132 2026-02-19T12:43:22.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.8132/$cluster-$name.asok' 2026-02-19T12:43:22.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.8132/$cluster-$name.asok' 2026-02-19T12:43:22.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:43:22.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-mapper/$name.log' 2026-02-19T12:43:22.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-mapper/$name.pid' 2026-02-19T12:43:22.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:43:22.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:43:22.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:43:22.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:43:22.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:43:22.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:43:22.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-mapper/1 2026-02-19T12:43:22.662 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:43:22.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:43:22.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=e8d2c67c-d0fb-410d-b2e1-8374392917aa --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-mapper/1 --osd-journal=td/osd-mapper/1/journal --chdir= --run-dir=td/osd-mapper '--admin-socket=/tmp/ceph-asok.8132/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-mapper/$name.log' '--pid-file=td/osd-mapper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:43:22.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-mapper/1/whoami 2026-02-19T12:43:22.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:43:22.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:43:22.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:43:22.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:43:22.692 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:22.687+0000 7fa2c78a1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:22.694 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:22.691+0000 7fa2c78a1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:22.695 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:22.691+0000 7fa2c78a1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:22.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:43:22.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:43:22.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:43:22.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:43:22.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:43:22.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:43:22.827 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:43:22.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:43:22.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:43:22.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:43:22.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:43:23.079 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:23.075+0000 7fa2c78a1a40 -1 Falling back to public interface 2026-02-19T12:43:23.692 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:23.687+0000 7fa2c78a1a40 -1 osd.1 24 log_to_monitors true 2026-02-19T12:43:24.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:43:24.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:43:24.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:43:24.001 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:43:24.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:43:24.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:43:24.168 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 30 up_thru 30 down_at 24 last_clean_interval [9,24) [v2:127.0.0.1:6810/3429927153,v1:127.0.0.1:6811/3429927153] [v2:127.0.0.1:6812/3429927153,v1:127.0.0.1:6813/3429927153] exists,up 444b5105-b47a-4bb2-93ac-f2857cdde544 2026-02-19T12:43:24.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:43:24.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:43:24.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:43:24.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-19T12:43:24.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: CEPH_ARGS='--fsid=e8d2c67c-d0fb-410d-b2e1-8374392917aa --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3' 2026-02-19T12:43:24.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: activate_osd td/osd-mapper 2 2026-02-19T12:43:24.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-mapper 2026-02-19T12:43:24.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:43:24.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T12:43:24.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:43:24.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-mapper/2 2026-02-19T12:43:24.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=e8d2c67c-d0fb-410d-b2e1-8374392917aa --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3' 2026-02-19T12:43:24.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:43:24.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:43:24.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:43:24.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-mapper/2' 2026-02-19T12:43:24.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-mapper/2/journal' 2026-02-19T12:43:24.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:43:24.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:43:24.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-mapper' 2026-02-19T12:43:24.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:43:24.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:43:24.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:43:24.173 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:43:24.173 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:24.173 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.8132 2026-02-19T12:43:24.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.8132/$cluster-$name.asok' 2026-02-19T12:43:24.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.8132/$cluster-$name.asok' 2026-02-19T12:43:24.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:43:24.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-mapper/$name.log' 2026-02-19T12:43:24.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-mapper/$name.pid' 2026-02-19T12:43:24.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:43:24.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:43:24.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:43:24.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:43:24.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:43:24.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:43:24.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-mapper/2 2026-02-19T12:43:24.174 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T12:43:24.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T12:43:24.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=e8d2c67c-d0fb-410d-b2e1-8374392917aa --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=5 --osd_scrub_retry_pg_state=5 --osd_scrub_retry_delay=3 --osd_pool_default_size=3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-mapper/2 --osd-journal=td/osd-mapper/2/journal --chdir= --run-dir=td/osd-mapper '--admin-socket=/tmp/ceph-asok.8132/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-mapper/$name.log' '--pid-file=td/osd-mapper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:43:24.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-mapper/2/whoami 2026-02-19T12:43:24.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T12:43:24.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:43:24.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:43:24.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:43:24.204 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:24.199+0000 7f27ccc4da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:24.206 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:24.203+0000 7f27ccc4da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:24.207 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:24.203+0000 7f27ccc4da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:24.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T12:43:24.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:43:24.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:43:24.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:43:24.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:43:24.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:43:24.342 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:43:24.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:43:24.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:43:24.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:43:24.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:43:24.603 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:24.599+0000 7f27ccc4da40 -1 Falling back to public interface 2026-02-19T12:43:25.312 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:25.307+0000 7f27ccc4da40 -1 osd.2 24 log_to_monitors true 2026-02-19T12:43:25.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:43:25.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:43:25.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:43:25.515 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:43:25.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:43:25.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:43:25.684 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 34 up_thru 34 down_at 24 last_clean_interval [13,24) [v2:127.0.0.1:6818/1283633833,v1:127.0.0.1:6819/1283633833] [v2:127.0.0.1:6820/1283633833,v1:127.0.0.1:6821/1283633833] exists,up 4fef2bbb-13d2-4c7e-a671-3d17e90dd637 2026-02-19T12:43:25.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:43:25.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:43:25.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:43:25.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:136: TEST_truncated_sna_record: sleep 1 2026-02-19T12:43:26.687 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: expr 3 - 1 2026-02-19T12:43:26.688 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: seq 0 2 2026-02-19T12:43:26.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-19T12:43:26.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:140: TEST_truncated_sna_record: timeout 60 ceph tell osd.0 version 2026-02-19T12:43:26.755 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T12:43:26.755 INFO:tasks.workunit.client.0.trial144.stdout: "version": "20.2.0-720-g2fc421c8", 2026-02-19T12:43:26.756 INFO:tasks.workunit.client.0.trial144.stdout: "release": "tentacle", 2026-02-19T12:43:26.756 INFO:tasks.workunit.client.0.trial144.stdout: "release_type": "stable" 2026-02-19T12:43:26.756 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:43:26.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-19T12:43:26.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:140: TEST_truncated_sna_record: timeout 60 ceph tell osd.1 version 2026-02-19T12:43:26.825 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T12:43:26.826 INFO:tasks.workunit.client.0.trial144.stdout: "version": "20.2.0-720-g2fc421c8", 2026-02-19T12:43:26.826 INFO:tasks.workunit.client.0.trial144.stdout: "release": "tentacle", 2026-02-19T12:43:26.826 INFO:tasks.workunit.client.0.trial144.stdout: "release_type": "stable" 2026-02-19T12:43:26.826 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:43:26.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-19T12:43:26.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:140: TEST_truncated_sna_record: timeout 60 ceph tell osd.2 version 2026-02-19T12:43:26.897 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T12:43:26.897 INFO:tasks.workunit.client.0.trial144.stdout: "version": "20.2.0-720-g2fc421c8", 2026-02-19T12:43:26.897 INFO:tasks.workunit.client.0.trial144.stdout: "release": "tentacle", 2026-02-19T12:43:26.897 INFO:tasks.workunit.client.0.trial144.stdout: "release_type": "stable" 2026-02-19T12:43:26.897 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:43:26.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:142: TEST_truncated_sna_record: rados --format json-pretty -p test listsnaps objxxx 2026-02-19T12:43:26.921 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T12:43:26.921 INFO:tasks.workunit.client.0.trial144.stdout: "name": "objxxx", 2026-02-19T12:43:26.921 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 5, 2026-02-19T12:43:26.921 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T12:43:26.921 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:26.921 INFO:tasks.workunit.client.0.trial144.stdout: "id": 2, 2026-02-19T12:43:26.921 INFO:tasks.workunit.client.0.trial144.stdout: "snapshots": [ 2026-02-19T12:43:26.922 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:26.922 INFO:tasks.workunit.client.0.trial144.stdout: "id": 1, 2026-02-19T12:43:26.922 INFO:tasks.workunit.client.0.trial144.stdout: "name": "snap01" 2026-02-19T12:43:26.922 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T12:43:26.922 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:26.922 INFO:tasks.workunit.client.0.trial144.stdout: "id": 2, 2026-02-19T12:43:26.922 INFO:tasks.workunit.client.0.trial144.stdout: "name": "snap02" 2026-02-19T12:43:26.922 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T12:43:26.922 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T12:43:26.922 INFO:tasks.workunit.client.0.trial144.stdout: "size": 5, 2026-02-19T12:43:26.922 INFO:tasks.workunit.client.0.trial144.stdout: "overlaps": [] 2026-02-19T12:43:26.922 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T12:43:26.923 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:26.923 INFO:tasks.workunit.client.0.trial144.stdout: "id": 3, 2026-02-19T12:43:26.923 INFO:tasks.workunit.client.0.trial144.stdout: "snapshots": [ 2026-02-19T12:43:26.923 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:26.923 INFO:tasks.workunit.client.0.trial144.stdout: "id": 3, 2026-02-19T12:43:26.923 INFO:tasks.workunit.client.0.trial144.stdout: "name": "snap13" 2026-02-19T12:43:26.923 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T12:43:26.923 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T12:43:26.923 INFO:tasks.workunit.client.0.trial144.stdout: "size": 5, 2026-02-19T12:43:26.923 INFO:tasks.workunit.client.0.trial144.stdout: "overlaps": [] 2026-02-19T12:43:26.923 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T12:43:26.923 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:26.923 INFO:tasks.workunit.client.0.trial144.stdout: "id": 5, 2026-02-19T12:43:26.924 INFO:tasks.workunit.client.0.trial144.stdout: "snapshots": [ 2026-02-19T12:43:26.924 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:26.924 INFO:tasks.workunit.client.0.trial144.stdout: "id": 4, 2026-02-19T12:43:26.924 INFO:tasks.workunit.client.0.trial144.stdout: "name": "snap24" 2026-02-19T12:43:26.924 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T12:43:26.924 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:26.924 INFO:tasks.workunit.client.0.trial144.stdout: "id": 5, 2026-02-19T12:43:26.924 INFO:tasks.workunit.client.0.trial144.stdout: "name": "snap25" 2026-02-19T12:43:26.924 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T12:43:26.924 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T12:43:26.924 INFO:tasks.workunit.client.0.trial144.stdout: "size": 5, 2026-02-19T12:43:26.924 INFO:tasks.workunit.client.0.trial144.stdout: "overlaps": [] 2026-02-19T12:43:26.924 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T12:43:26.925 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T12:43:26.925 INFO:tasks.workunit.client.0.trial144.stdout: "id": "head", 2026-02-19T12:43:26.925 INFO:tasks.workunit.client.0.trial144.stdout: "snapshots": [], 2026-02-19T12:43:26.925 INFO:tasks.workunit.client.0.trial144.stdout: "size": 6 2026-02-19T12:43:26.925 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T12:43:26.925 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T12:43:26.925 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:43:26.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:145: TEST_truncated_sna_record: ceph osd unset nodeep-scrub 2026-02-19T12:43:27.157 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is unset 2026-02-19T12:43:27.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:146: TEST_truncated_sna_record: ceph osd unset noscrub 2026-02-19T12:43:27.372 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is unset 2026-02-19T12:43:27.383 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:149: TEST_truncated_sna_record: ceph --format=json-pretty osd map test objxxx 2026-02-19T12:43:27.383 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:149: TEST_truncated_sna_record: jq -r '.up[0]' 2026-02-19T12:43:27.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:149: TEST_truncated_sna_record: local cur_prim=1 2026-02-19T12:43:27.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:150: TEST_truncated_sna_record: ceph pg dump pgs 2026-02-19T12:43:27.689 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:43:27.689 INFO:tasks.workunit.client.0.trial144.stdout:1.3 4 0 0 0 0 21 0 0 7 0 7 active+clean 2026-02-19T12:43:25.671588+0000 21'7 36:71 [1,2,0] 1 [1,2,0] 1 21'7 2026-02-19T12:43:08.129189+0000 21'7 2026-02-19T12:43:08.129189+0000 0 1 periodic scrub scheduled @ 2026-02-20T12:43:08.129189+0000 4 0 2026-02-19T12:43:27.689 INFO:tasks.workunit.client.0.trial144.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:25.671666+0000 0'0 36:62 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-19T12:43:01.405630+0000 0'0 2026-02-19T12:43:01.405630+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:01.405630+0000 0 0 2026-02-19T12:43:27.689 INFO:tasks.workunit.client.0.trial144.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:25.572456+0000 0'0 36:51 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-19T12:43:01.405630+0000 0'0 2026-02-19T12:43:01.405630+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:01.405630+0000 0 0 2026-02-19T12:43:27.689 INFO:tasks.workunit.client.0.trial144.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:25.671595+0000 0'0 36:61 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T12:43:01.405630+0000 0'0 2026-02-19T12:43:01.405630+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:01.405630+0000 0 0 2026-02-19T12:43:27.689 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:27.689 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:43:27.690 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:43:27.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:151: TEST_truncated_sna_record: sleep 2 2026-02-19T12:43:29.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:152: TEST_truncated_sna_record: ceph pg 1.3 deep-scrub 2026-02-19T12:43:29.765 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T12:43:29.765 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T12:43:29.765 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T12:43:29.765 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T12:43:29.765 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:43:29.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:153: TEST_truncated_sna_record: sleep 5 2026-02-19T12:43:34.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:154: TEST_truncated_sna_record: ceph pg dump pgs 2026-02-19T12:43:34.920 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:43:34.921 INFO:tasks.workunit.client.0.trial144.stdout:1.3 4 0 0 0 0 21 0 0 7 0 7 active+clean 2026-02-19T12:43:30.642814+0000 21'7 38:85 [1,2,0] 1 [1,2,0] 1 21'7 2026-02-19T12:43:30.642772+0000 21'7 2026-02-19T12:43:30.642772+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:30.642772+0000 4 0 2026-02-19T12:43:34.921 INFO:tasks.workunit.client.0.trial144.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:25.671666+0000 0'0 38:67 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-19T12:43:01.405630+0000 0'0 2026-02-19T12:43:01.405630+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:01.405630+0000 0 0 2026-02-19T12:43:34.921 INFO:tasks.workunit.client.0.trial144.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:25.572456+0000 0'0 38:55 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-19T12:43:01.405630+0000 0'0 2026-02-19T12:43:01.405630+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:01.405630+0000 0 0 2026-02-19T12:43:34.921 INFO:tasks.workunit.client.0.trial144.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:25.671595+0000 0'0 38:67 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T12:43:01.405630+0000 0'0 2026-02-19T12:43:01.405630+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:01.405630+0000 0 0 2026-02-19T12:43:34.921 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:34.921 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:43:34.921 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:43:34.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:155: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-19T12:43:34.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:155: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2026-02-19T12:43:34.930 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:43:30.639+0000 7fa2a381b640 -1 log_channel(cluster) log [ERR] : osd.1 found snap mapper error on pg 1.3 oid 1:f5ae0ba1:::objxxx:3 snaps in mapper: {}, oi: {3} ...repaired 2026-02-19T12:43:34.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:156: TEST_truncated_sna_record: grep -a -q ERR td/osd-mapper/osd.1.log 2026-02-19T12:43:34.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:160: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2026-02-19T12:43:34.934 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:160: TEST_truncated_sna_record: wc -l 2026-02-19T12:43:34.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:160: TEST_truncated_sna_record: local prev_err_cnt=1 2026-02-19T12:43:34.936 INFO:tasks.workunit.client.0.trial144.stdout:prev count: 1 2026-02-19T12:43:34.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:161: TEST_truncated_sna_record: echo 'prev count: 1' 2026-02-19T12:43:34.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:164: TEST_truncated_sna_record: ceph pg 1.3 deep-scrub 2026-02-19T12:43:35.004 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T12:43:35.004 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T12:43:35.004 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T12:43:35.004 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T12:43:35.004 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:43:35.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:165: TEST_truncated_sna_record: sleep 5 2026-02-19T12:43:40.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:166: TEST_truncated_sna_record: ceph pg dump pgs 2026-02-19T12:43:40.155 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:43:40.155 INFO:tasks.workunit.client.0.trial144.stdout:1.3 4 0 0 0 0 21 0 0 7 0 7 active+clean 2026-02-19T12:43:35.683041+0000 21'7 38:94 [1,2,0] 1 [1,2,0] 1 21'7 2026-02-19T12:43:35.682996+0000 21'7 2026-02-19T12:43:35.682996+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:35.682996+0000 4 0 2026-02-19T12:43:40.156 INFO:tasks.workunit.client.0.trial144.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:25.671666+0000 0'0 38:68 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-19T12:43:01.405630+0000 0'0 2026-02-19T12:43:01.405630+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:01.405630+0000 0 0 2026-02-19T12:43:40.156 INFO:tasks.workunit.client.0.trial144.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:25.572456+0000 0'0 38:56 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-19T12:43:01.405630+0000 0'0 2026-02-19T12:43:01.405630+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:01.405630+0000 0 0 2026-02-19T12:43:40.156 INFO:tasks.workunit.client.0.trial144.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:25.671595+0000 0'0 38:68 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T12:43:01.405630+0000 0'0 2026-02-19T12:43:01.405630+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:43:01.405630+0000 0 0 2026-02-19T12:43:40.156 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:40.156 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:43:40.156 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:43:40.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:167: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-19T12:43:40.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:167: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2026-02-19T12:43:40.165 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:43:30.639+0000 7fa2a381b640 -1 log_channel(cluster) log [ERR] : osd.1 found snap mapper error on pg 1.3 oid 1:f5ae0ba1:::objxxx:3 snaps in mapper: {}, oi: {3} ...repaired 2026-02-19T12:43:40.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:168: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2026-02-19T12:43:40.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:168: TEST_truncated_sna_record: wc -l 2026-02-19T12:43:40.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:168: TEST_truncated_sna_record: local current_err_cnt=1 2026-02-19T12:43:40.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:169: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-19T12:43:40.169 INFO:tasks.workunit.client.0.trial144.stdout:current count: 1 2026-02-19T12:43:40.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:169: TEST_truncated_sna_record: echo 'current count: 1' 2026-02-19T12:43:40.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:170: TEST_truncated_sna_record: (( current_err_cnt == prev_err_cnt )) 2026-02-19T12:43:40.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:171: TEST_truncated_sna_record: kill_daemons td/osd-mapper TERM osd 2026-02-19T12:43:40.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:43:40.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:43:40.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:43:40.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:43:40.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:43:40.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:43:40.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:172: TEST_truncated_sna_record: kvdir=td/osd-mapper/1 2026-02-19T12:43:40.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:173: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-19T12:43:40.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:173: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2026-02-19T12:43:40.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:173: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_' 2026-02-19T12:43:40.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:174: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-19T12:43:40.705 INFO:tasks.workunit.client.0.trial144.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:40.705 INFO:tasks.workunit.client.0.trial144.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-19T12:43:40.705 INFO:tasks.workunit.client.0.trial144.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2026-02-19T12:43:40.705 INFO:tasks.workunit.client.0.trial144.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-19T12:43:40.705 INFO:tasks.workunit.client.0.trial144.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:40.705 INFO:tasks.workunit.client.0.trial144.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-19T12:43:40.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2026-02-19T12:43:40.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_000000000000000[0-9]_000000000000000' 2026-02-19T12:43:40.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-19T12:43:40.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: wc -l 2026-02-19T12:43:40.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: local num_sna_full=5 2026-02-19T12:43:40.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:177: TEST_truncated_sna_record: (( num_sna_full == num_sna_b4 )) 2026-02-19T12:43:40.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:178: TEST_truncated_sna_record: return 0 2026-02-19T12:43:40.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:24: run: teardown td/osd-mapper 2026-02-19T12:43:40.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-mapper 2026-02-19T12:43:40.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:43:40.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-mapper KILL 2026-02-19T12:43:40.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:43:40.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:43:40.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:43:40.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:43:40.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:43:41.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:43:41.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:43:41.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:43:41.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:43:41.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:43:41.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:43:41.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:43:41.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:43:41.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:43:41.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:43:41.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:43:41.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:43:41.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:43:41.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-mapper 2026-02-19T12:43:41.054 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:43:41.054 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:41.054 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.8132 2026-02-19T12:43:41.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.8132 2026-02-19T12:43:41.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:43:41.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:43:41.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-19T12:43:41.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/osd-mapper 0 2026-02-19T12:43:41.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-mapper 2026-02-19T12:43:41.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-19T12:43:41.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-mapper KILL 2026-02-19T12:43:41.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:43:41.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:43:41.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:43:41.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:43:41.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:43:41.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:43:41.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:43:41.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:43:41.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:43:41.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:43:41.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:43:41.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:43:41.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:43:41.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:43:41.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:43:41.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:43:41.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:43:41.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-19T12:43:41.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-mapper 2026-02-19T12:43:41.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:43:41.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:41.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.8132 2026-02-19T12:43:41.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.8132 2026-02-19T12:43:41.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:43:41.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:43:41.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-19T12:43:41.068 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-19T12:43:41.068 DEBUG:teuthology.orchestra.run.trial144:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-19T12:43:41.117 INFO:tasks.workunit:Running workunit scrub/osd-recovery-scrub.sh... 2026-02-19T12:43:41.117 DEBUG:teuthology.orchestra.run.trial144:workunit test scrub/osd-recovery-scrub.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=624b9a7bcb2064ce17206482df6a1da007ff7675 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh 2026-02-19T12:43:41.170 INFO:tasks.workunit.client.0.trial144.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-19T12:43:41.173 INFO:tasks.workunit.client.0.trial144.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-19T12:43:41.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-19T12:43:41.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-19T12:43:41.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-19T12:43:41.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-19T12:43:41.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-19T12:43:41.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-19T12:43:41.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-19T12:43:41.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-19T12:43:41.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/osd-recovery-scrub 2026-02-19T12:43:41.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:20: run: local dir=td/osd-recovery-scrub 2026-02-19T12:43:41.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:21: run: shift 2026-02-19T12:43:41.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:23: run: export CEPH_MON=127.0.0.1:7124 2026-02-19T12:43:41.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:23: run: CEPH_MON=127.0.0.1:7124 2026-02-19T12:43:41.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:24: run: export CEPH_ARGS 2026-02-19T12:43:41.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:25: run: uuidgen 2026-02-19T12:43:41.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:25: run: CEPH_ARGS+='--fsid=b4e8090f-6e94-4333-b3c7-80422c6cd54d --auth-supported=none ' 2026-02-19T12:43:41.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:26: run: CEPH_ARGS+='--mon-host=127.0.0.1:7124 ' 2026-02-19T12:43:41.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:27: run: CEPH_ARGS+='--osd-op-queue=wpq ' 2026-02-19T12:43:41.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:29: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-19T12:43:41.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:30: run: set 2026-02-19T12:43:41.177 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-19T12:43:41.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:30: run: local funcs=TEST_recovery_scrub_1 2026-02-19T12:43:41.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:31: run: for func in $funcs 2026-02-19T12:43:41.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:32: run: TEST_recovery_scrub_1 td/osd-recovery-scrub 2026-02-19T12:43:41.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:39: TEST_recovery_scrub_1: local dir=td/osd-recovery-scrub 2026-02-19T12:43:41.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:40: TEST_recovery_scrub_1: local poolname=test 2026-02-19T12:43:41.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:42: TEST_recovery_scrub_1: TESTDATA=testdata.26613 2026-02-19T12:43:41.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:43: TEST_recovery_scrub_1: OSDS=4 2026-02-19T12:43:41.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:44: TEST_recovery_scrub_1: PGS=1 2026-02-19T12:43:41.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:45: TEST_recovery_scrub_1: OBJECTS=100 2026-02-19T12:43:41.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:46: TEST_recovery_scrub_1: ERRORS=0 2026-02-19T12:43:41.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:48: TEST_recovery_scrub_1: setup td/osd-recovery-scrub 2026-02-19T12:43:41.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-recovery-scrub 2026-02-19T12:43:41.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-recovery-scrub 2026-02-19T12:43:41.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-recovery-scrub 2026-02-19T12:43:41.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:43:41.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-recovery-scrub KILL 2026-02-19T12:43:41.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:43:41.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:43:41.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:43:41.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:43:41.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:43:41.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:43:41.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:43:41.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:43:41.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:43:41.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:43:41.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:43:41.183 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:43:41.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:43:41.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:43:41.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:43:41.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:43:41.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:43:41.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:43:41.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-recovery-scrub 2026-02-19T12:43:41.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:43:41.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:41.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.26613 2026-02-19T12:43:41.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.26613 2026-02-19T12:43:41.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:43:41.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:43:41.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-recovery-scrub 2026-02-19T12:43:41.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:43:41.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:41.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.26613 2026-02-19T12:43:41.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.26613 2026-02-19T12:43:41.193 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:43:41.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-19T12:43:41.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-19T12:43:41.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:43:41.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-recovery-scrub 1' TERM HUP INT 2026-02-19T12:43:41.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:49: TEST_recovery_scrub_1: run_mon td/osd-recovery-scrub a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-19T12:43:41.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-recovery-scrub 2026-02-19T12:43:41.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T12:43:41.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T12:43:41.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T12:43:41.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-recovery-scrub/a 2026-02-19T12:43:41.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-recovery-scrub/a --run-dir=td/osd-recovery-scrub --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-19T12:43:41.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T12:43:41.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:43:41.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:43:41.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:43:41.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:41.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.26613 2026-02-19T12:43:41.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.26613/$cluster-$name.asok' 2026-02-19T12:43:41.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-recovery-scrub/a '--log-file=td/osd-recovery-scrub/$name.log' '--admin-socket=/tmp/ceph-asok.26613/$cluster-$name.asok' --mon-cluster-log-file=td/osd-recovery-scrub/log --run-dir=td/osd-recovery-scrub '--pid-file=td/osd-recovery-scrub/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-19T12:43:41.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T12:43:41.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T12:43:41.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:43:41.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:43:41.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T12:43:41.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T12:43:41.251 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:43:41.251 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:43:41.252 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:43:41.252 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:43:41.252 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:41.252 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.26613 2026-02-19T12:43:41.252 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.26613/ceph-mon.a.asok 2026-02-19T12:43:41.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:43:41.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.26613/ceph-mon.a.asok config get fsid 2026-02-19T12:43:41.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T12:43:41.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:43:41.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:43:41.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T12:43:41.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T12:43:41.308 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:43:41.308 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:43:41.308 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:43:41.308 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:43:41.308 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:41.308 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.26613 2026-02-19T12:43:41.308 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.26613/ceph-mon.a.asok 2026-02-19T12:43:41.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:43:41.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.26613/ceph-mon.a.asok config get mon_host 2026-02-19T12:43:41.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:50: TEST_recovery_scrub_1: run_mgr td/osd-recovery-scrub x 2026-02-19T12:43:41.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-recovery-scrub 2026-02-19T12:43:41.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T12:43:41.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T12:43:41.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T12:43:41.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-recovery-scrub/x 2026-02-19T12:43:41.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T12:43:41.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T12:43:41.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:43:41.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:43:41.456 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:43:41.456 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:41.456 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.26613 2026-02-19T12:43:41.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.26613/$cluster-$name.asok' 2026-02-19T12:43:41.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:43:41.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-recovery-scrub/x '--log-file=td/osd-recovery-scrub/$name.log' '--admin-socket=/tmp/ceph-asok.26613/$cluster-$name.asok' --run-dir=td/osd-recovery-scrub '--pid-file=td/osd-recovery-scrub/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:43:41.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:51: TEST_recovery_scrub_1: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 ' 2026-02-19T12:43:41.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:52: TEST_recovery_scrub_1: ceph_osd_args+='--osd_scrub_backoff_ratio=0 ' 2026-02-19T12:43:41.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:53: TEST_recovery_scrub_1: ceph_osd_args+='--osd_stats_update_period_not_scrubbing=3 ' 2026-02-19T12:43:41.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:54: TEST_recovery_scrub_1: ceph_osd_args+=--osd_stats_update_period_scrubbing=2 2026-02-19T12:43:41.477 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: expr 4 - 1 2026-02-19T12:43:41.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: seq 0 3 2026-02-19T12:43:41.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T12:43:41.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 0 --osd_scrub_during_recovery=false 2026-02-19T12:43:41.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-recovery-scrub 2026-02-19T12:43:41.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:43:41.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T12:43:41.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:43:41.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-recovery-scrub/0 2026-02-19T12:43:41.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=b4e8090f-6e94-4333-b3c7-80422c6cd54d --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2026-02-19T12:43:41.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:43:41.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:43:41.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:43:41.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/0' 2026-02-19T12:43:41.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/0/journal' 2026-02-19T12:43:41.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:43:41.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:43:41.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2026-02-19T12:43:41.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:43:41.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:43:41.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:43:41.482 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:43:41.482 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:41.483 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.26613 2026-02-19T12:43:41.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.26613/$cluster-$name.asok' 2026-02-19T12:43:41.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.26613/$cluster-$name.asok' 2026-02-19T12:43:41.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:43:41.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:43:41.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:43:41.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2026-02-19T12:43:41.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2026-02-19T12:43:41.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:43:41.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:43:41.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:43:41.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:43:41.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:43:41.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2026-02-19T12:43:41.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-recovery-scrub/0 2026-02-19T12:43:41.485 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:43:41.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8a98ac62-dbf2-430e-837e-79fed3e7c99a 2026-02-19T12:43:41.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8a98ac62-dbf2-430e-837e-79fed3e7c99a' 2026-02-19T12:43:41.486 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 8a98ac62-dbf2-430e-837e-79fed3e7c99a 2026-02-19T12:43:41.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:43:41.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB9BZdpmjhcHRAAUwNuaCq9hNN7oxE0hxO9gw== 2026-02-19T12:43:41.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB9BZdpmjhcHRAAUwNuaCq9hNN7oxE0hxO9gw=="}' 2026-02-19T12:43:41.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8a98ac62-dbf2-430e-837e-79fed3e7c99a -i td/osd-recovery-scrub/0/new.json 2026-02-19T12:43:41.583 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:43:41.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-recovery-scrub/0/new.json 2026-02-19T12:43:41.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=b4e8090f-6e94-4333-b3c7-80422c6cd54d --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/0 --osd-journal=td/osd-recovery-scrub/0/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.26613/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQB9BZdpmjhcHRAAUwNuaCq9hNN7oxE0hxO9gw== --osd-uuid 8a98ac62-dbf2-430e-837e-79fed3e7c99a 2026-02-19T12:43:41.610 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:41.607+0000 7fa5b8418a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:41.612 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:41.607+0000 7fa5b8418a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:41.613 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:41.607+0000 7fa5b8418a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:41.613 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:41.611+0000 7fa5b8418a40 -1 bdev(0x555ea8966c00 td/osd-recovery-scrub/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:43:41.613 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:41.611+0000 7fa5b8418a40 -1 bluestore(td/osd-recovery-scrub/0) _read_fsid unparsable uuid 2026-02-19T12:43:42.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-recovery-scrub/0/keyring 2026-02-19T12:43:42.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:43:42.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T12:43:42.348 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:43:42.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-recovery-scrub/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:43:42.550 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:43:42.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T12:43:42.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=b4e8090f-6e94-4333-b3c7-80422c6cd54d --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/0 --osd-journal=td/osd-recovery-scrub/0/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.26613/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2026-02-19T12:43:42.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:43:42.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:43:42.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:43:42.575 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:42.571+0000 7f62ffc38a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:42.577 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:42.575+0000 7f62ffc38a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:42.578 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:42.575+0000 7f62ffc38a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:42.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T12:43:42.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:43:42.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:43:42.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:43:42.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:43:42.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:43:42.711 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:43:42.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:43:42.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:43:42.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:43:42.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:43:42.986 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:42.983+0000 7f62ffc38a40 -1 Falling back to public interface 2026-02-19T12:43:43.588 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:43.583+0000 7f62ffc38a40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:43:43.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:43:43.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:43:43.871 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:43:43.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:43:43.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:43:43.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:43:44.032 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/331126818,v1:127.0.0.1:6803/331126818] [v2:127.0.0.1:6804/331126818,v1:127.0.0.1:6805/331126818] exists,up 8a98ac62-dbf2-430e-837e-79fed3e7c99a 2026-02-19T12:43:44.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:43:44.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:43:44.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:43:44.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T12:43:44.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 1 --osd_scrub_during_recovery=false 2026-02-19T12:43:44.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-recovery-scrub 2026-02-19T12:43:44.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:43:44.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T12:43:44.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:43:44.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-recovery-scrub/1 2026-02-19T12:43:44.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=b4e8090f-6e94-4333-b3c7-80422c6cd54d --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2026-02-19T12:43:44.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:43:44.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:43:44.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:43:44.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/1' 2026-02-19T12:43:44.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/1/journal' 2026-02-19T12:43:44.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:43:44.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:43:44.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2026-02-19T12:43:44.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:43:44.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:43:44.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:43:44.038 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:43:44.038 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:44.038 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.26613 2026-02-19T12:43:44.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.26613/$cluster-$name.asok' 2026-02-19T12:43:44.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.26613/$cluster-$name.asok' 2026-02-19T12:43:44.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:43:44.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:43:44.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:43:44.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2026-02-19T12:43:44.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2026-02-19T12:43:44.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:43:44.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:43:44.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:43:44.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:43:44.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:43:44.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2026-02-19T12:43:44.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-recovery-scrub/1 2026-02-19T12:43:44.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:43:44.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=96977216-d72f-4e77-b3da-5ee594031e75 2026-02-19T12:43:44.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 96977216-d72f-4e77-b3da-5ee594031e75' 2026-02-19T12:43:44.040 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 96977216-d72f-4e77-b3da-5ee594031e75 2026-02-19T12:43:44.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:43:44.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCABZdpr+PKAhAAoNyTnwtucR2PEicN8TAJFw== 2026-02-19T12:43:44.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCABZdpr+PKAhAAoNyTnwtucR2PEicN8TAJFw=="}' 2026-02-19T12:43:44.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 96977216-d72f-4e77-b3da-5ee594031e75 -i td/osd-recovery-scrub/1/new.json 2026-02-19T12:43:44.203 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:43:44.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-recovery-scrub/1/new.json 2026-02-19T12:43:44.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=b4e8090f-6e94-4333-b3c7-80422c6cd54d --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/1 --osd-journal=td/osd-recovery-scrub/1/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.26613/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQCABZdpr+PKAhAAoNyTnwtucR2PEicN8TAJFw== --osd-uuid 96977216-d72f-4e77-b3da-5ee594031e75 2026-02-19T12:43:44.240 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:44.235+0000 7ff7dc92fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:44.242 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:44.239+0000 7ff7dc92fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:44.243 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:44.239+0000 7ff7dc92fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:44.243 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:44.239+0000 7ff7dc92fa40 -1 bdev(0x5623dafe4c00 td/osd-recovery-scrub/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:43:44.244 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:44.239+0000 7ff7dc92fa40 -1 bluestore(td/osd-recovery-scrub/1) _read_fsid unparsable uuid 2026-02-19T12:43:44.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-recovery-scrub/1/keyring 2026-02-19T12:43:44.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:43:44.869 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:43:44.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T12:43:44.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-recovery-scrub/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:43:45.072 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:43:45.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T12:43:45.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=b4e8090f-6e94-4333-b3c7-80422c6cd54d --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/1 --osd-journal=td/osd-recovery-scrub/1/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.26613/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2026-02-19T12:43:45.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:43:45.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:43:45.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:43:45.097 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:45.095+0000 7f5b1739fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:45.099 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:45.095+0000 7f5b1739fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:45.100 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:45.095+0000 7f5b1739fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:45.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T12:43:45.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:43:45.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:43:45.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:43:45.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:43:45.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:43:45.227 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:43:45.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:43:45.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:43:45.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:43:45.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:43:45.470 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:45.467+0000 7f5b1739fa40 -1 Falling back to public interface 2026-02-19T12:43:46.082 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:46.079+0000 7f5b1739fa40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:43:46.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:43:46.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:43:46.388 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:43:46.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:43:46.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:43:46.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:43:46.546 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2493579557,v1:127.0.0.1:6811/2493579557] [v2:127.0.0.1:6812/2493579557,v1:127.0.0.1:6813/2493579557] exists,up 96977216-d72f-4e77-b3da-5ee594031e75 2026-02-19T12:43:46.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:43:46.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:43:46.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:43:46.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T12:43:46.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 2 --osd_scrub_during_recovery=false 2026-02-19T12:43:46.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-recovery-scrub 2026-02-19T12:43:46.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:43:46.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T12:43:46.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:43:46.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-recovery-scrub/2 2026-02-19T12:43:46.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=b4e8090f-6e94-4333-b3c7-80422c6cd54d --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2026-02-19T12:43:46.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:43:46.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:43:46.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:43:46.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/2' 2026-02-19T12:43:46.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/2/journal' 2026-02-19T12:43:46.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:43:46.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:43:46.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2026-02-19T12:43:46.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:43:46.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:43:46.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:43:46.551 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:43:46.551 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:46.551 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.26613 2026-02-19T12:43:46.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.26613/$cluster-$name.asok' 2026-02-19T12:43:46.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.26613/$cluster-$name.asok' 2026-02-19T12:43:46.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:43:46.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:43:46.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:43:46.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2026-02-19T12:43:46.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2026-02-19T12:43:46.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:43:46.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:43:46.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:43:46.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:43:46.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:43:46.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2026-02-19T12:43:46.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-recovery-scrub/2 2026-02-19T12:43:46.553 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:43:46.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=79aa8001-0d5d-4531-bef3-ef2cc47d375a 2026-02-19T12:43:46.553 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 79aa8001-0d5d-4531-bef3-ef2cc47d375a 2026-02-19T12:43:46.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 79aa8001-0d5d-4531-bef3-ef2cc47d375a' 2026-02-19T12:43:46.554 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:43:46.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCCBZdpov+AIRAAV19WzrGUj3RhgUGPmtfX6Q== 2026-02-19T12:43:46.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCCBZdpov+AIRAAV19WzrGUj3RhgUGPmtfX6Q=="}' 2026-02-19T12:43:46.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 79aa8001-0d5d-4531-bef3-ef2cc47d375a -i td/osd-recovery-scrub/2/new.json 2026-02-19T12:43:46.726 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:43:46.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-recovery-scrub/2/new.json 2026-02-19T12:43:46.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=b4e8090f-6e94-4333-b3c7-80422c6cd54d --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/2 --osd-journal=td/osd-recovery-scrub/2/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.26613/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQCCBZdpov+AIRAAV19WzrGUj3RhgUGPmtfX6Q== --osd-uuid 79aa8001-0d5d-4531-bef3-ef2cc47d375a 2026-02-19T12:43:46.754 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:46.751+0000 7f5f24ec1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:46.755 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:46.751+0000 7f5f24ec1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:46.756 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:46.751+0000 7f5f24ec1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:46.757 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:46.751+0000 7f5f24ec1a40 -1 bdev(0x55b94bbeec00 td/osd-recovery-scrub/2/block) open stat got: (1) Operation not permitted 2026-02-19T12:43:46.757 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:46.755+0000 7f5f24ec1a40 -1 bluestore(td/osd-recovery-scrub/2) _read_fsid unparsable uuid 2026-02-19T12:43:47.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-recovery-scrub/2/keyring 2026-02-19T12:43:47.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:43:47.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T12:43:47.394 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T12:43:47.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-recovery-scrub/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:43:47.592 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T12:43:47.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T12:43:47.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=b4e8090f-6e94-4333-b3c7-80422c6cd54d --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/2 --osd-journal=td/osd-recovery-scrub/2/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.26613/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2026-02-19T12:43:47.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:43:47.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:43:47.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:43:47.608 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:47.603+0000 7f48e25b1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:47.609 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:47.607+0000 7f48e25b1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:47.610 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:47.607+0000 7f48e25b1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:47.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T12:43:47.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:43:47.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:43:47.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:43:47.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:43:47.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:43:47.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:43:47.750 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:43:47.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:43:47.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:43:47.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:43:48.007 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:48.003+0000 7f48e25b1a40 -1 Falling back to public interface 2026-02-19T12:43:48.591 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:48.587+0000 7f48e25b1a40 -1 osd.2 0 log_to_monitors true 2026-02-19T12:43:48.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:43:48.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:43:48.908 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:43:48.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:43:48.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:43:48.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:43:49.068 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3118706212,v1:127.0.0.1:6819/3118706212] [v2:127.0.0.1:6820/3118706212,v1:127.0.0.1:6821/3118706212] exists,up 79aa8001-0d5d-4531-bef3-ef2cc47d375a 2026-02-19T12:43:49.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:43:49.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:43:49.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:43:49.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T12:43:49.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 3 --osd_scrub_during_recovery=false 2026-02-19T12:43:49.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-recovery-scrub 2026-02-19T12:43:49.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:43:49.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T12:43:49.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:43:49.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-recovery-scrub/3 2026-02-19T12:43:49.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=b4e8090f-6e94-4333-b3c7-80422c6cd54d --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/3' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/3/journal' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.26613 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.26613/$cluster-$name.asok' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.26613/$cluster-$name.asok' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2026-02-19T12:43:49.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-recovery-scrub/3 2026-02-19T12:43:49.072 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:43:49.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1f2f8149-857c-4eaa-9d17-90fde38626e1 2026-02-19T12:43:49.072 INFO:tasks.workunit.client.0.trial144.stdout:add osd3 1f2f8149-857c-4eaa-9d17-90fde38626e1 2026-02-19T12:43:49.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 1f2f8149-857c-4eaa-9d17-90fde38626e1' 2026-02-19T12:43:49.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:43:49.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCFBZdpwlPwBBAA6t4OSDd49wLr+yjeQDgvcg== 2026-02-19T12:43:49.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCFBZdpwlPwBBAA6t4OSDd49wLr+yjeQDgvcg=="}' 2026-02-19T12:43:49.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1f2f8149-857c-4eaa-9d17-90fde38626e1 -i td/osd-recovery-scrub/3/new.json 2026-02-19T12:43:49.238 INFO:tasks.workunit.client.0.trial144.stdout:3 2026-02-19T12:43:49.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-recovery-scrub/3/new.json 2026-02-19T12:43:49.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=b4e8090f-6e94-4333-b3c7-80422c6cd54d --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/3 --osd-journal=td/osd-recovery-scrub/3/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.26613/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQCFBZdpwlPwBBAA6t4OSDd49wLr+yjeQDgvcg== --osd-uuid 1f2f8149-857c-4eaa-9d17-90fde38626e1 2026-02-19T12:43:49.266 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:49.263+0000 7f6c33d71a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:49.267 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:49.263+0000 7f6c33d71a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:49.268 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:49.263+0000 7f6c33d71a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:49.268 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:49.263+0000 7f6c33d71a40 -1 bdev(0x55e17bef0c00 td/osd-recovery-scrub/3/block) open stat got: (1) Operation not permitted 2026-02-19T12:43:49.269 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:49.263+0000 7f6c33d71a40 -1 bluestore(td/osd-recovery-scrub/3) _read_fsid unparsable uuid 2026-02-19T12:43:49.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-recovery-scrub/3/keyring 2026-02-19T12:43:49.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:43:49.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T12:43:49.912 INFO:tasks.workunit.client.0.trial144.stdout:adding osd3 key to auth repository 2026-02-19T12:43:49.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-recovery-scrub/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:43:50.126 INFO:tasks.workunit.client.0.trial144.stdout:start osd.3 2026-02-19T12:43:50.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T12:43:50.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=b4e8090f-6e94-4333-b3c7-80422c6cd54d --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/3 --osd-journal=td/osd-recovery-scrub/3/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.26613/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2026-02-19T12:43:50.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:43:50.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:43:50.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:43:50.151 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:50.147+0000 7f4420ba6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:50.153 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:50.151+0000 7f4420ba6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:50.154 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:50.151+0000 7f4420ba6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:43:50.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T12:43:50.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:43:50.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:43:50.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:43:50.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:43:50.287 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:43:50.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:43:50.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:43:50.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:43:50.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:43:50.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:43:50.543 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:50.539+0000 7f4420ba6a40 -1 Falling back to public interface 2026-02-19T12:43:51.144 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:43:51.139+0000 7f4420ba6a40 -1 osd.3 0 log_to_monitors true 2026-02-19T12:43:51.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:43:51.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:43:51.447 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:43:51.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:43:51.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:43:51.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:43:51.606 INFO:tasks.workunit.client.0.trial144.stdout:osd.3 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1893445334,v1:127.0.0.1:6827/1893445334] [v2:127.0.0.1:6828/1893445334,v1:127.0.0.1:6829/1893445334] exists,up 1f2f8149-857c-4eaa-9d17-90fde38626e1 2026-02-19T12:43:51.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:43:51.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:43:51.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:43:51.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:61: TEST_recovery_scrub_1: create_pool test 1 1 2026-02-19T12:43:51.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-19T12:43:51.810 INFO:tasks.workunit.client.0.trial144.stderr:pool 'test' created 2026-02-19T12:43:51.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:43:52.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:62: TEST_recovery_scrub_1: wait_for_clean 2026-02-19T12:43:52.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:43:52.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:43:52.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:43:52.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:43:52.823 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:43:52.823 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:43:52.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:43:52.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:43:52.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:43:52.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:43:52.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:43:52.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:43:52.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:43:52.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:43:52.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:43:53.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:43:53.084 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:43:53.084 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:43:53.084 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T12:43:53.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:43:53.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:43:53.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:43:53.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T12:43:53.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T12:43:53.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T12:43:53.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:43:53.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:43:53.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-19T12:43:53.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-19T12:43:53.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705667' 2026-02-19T12:43:53.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:43:53.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:43:53.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574850 2026-02-19T12:43:53.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574850 2026-02-19T12:43:53.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705667 2-55834574850' 2026-02-19T12:43:53.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:43:53.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:43:53.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444034 2026-02-19T12:43:53.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444034 2026-02-19T12:43:53.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705667 2-55834574850 3-73014444034' 2026-02-19T12:43:53.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:43:53.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T12:43:53.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:43:53.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:43:53.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T12:43:53.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:43:53.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T12:43:53.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T12:43:53.376 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T12:43:53.376 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:43:53.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T12:43:53.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:43:54.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:43:54.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:43:54.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-19T12:43:54.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:43:54.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-19T12:43:54.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:43:54.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:43:54.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705667 2026-02-19T12:43:54.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:43:54.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705667 2026-02-19T12:43:54.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-19T12:43:54.698 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705667 2026-02-19T12:43:54.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:43:54.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705667 -lt 38654705667 2026-02-19T12:43:54.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:43:54.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574850 2026-02-19T12:43:54.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:43:54.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:43:54.862 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574850 2026-02-19T12:43:54.862 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:43:54.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574850 2026-02-19T12:43:54.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574850' 2026-02-19T12:43:54.864 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574850 2026-02-19T12:43:54.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:43:55.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574850 -lt 55834574850 2026-02-19T12:43:55.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:43:55.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-73014444034 2026-02-19T12:43:55.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:43:55.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:43:55.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-73014444034 2026-02-19T12:43:55.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:43:55.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444034 2026-02-19T12:43:55.036 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.3 seq 73014444034 2026-02-19T12:43:55.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 73014444034' 2026-02-19T12:43:55.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:43:55.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444034 -lt 73014444034 2026-02-19T12:43:55.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:43:55.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:43:55.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:43:55.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:43:55.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:43:55.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:43:55.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:43:55.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:43:55.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:43:55.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:43:55.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:43:55.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:43:55.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:43:55.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:43:55.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:43:55.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:43:55.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:43:55.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:43:55.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: ceph osd dump 2026-02-19T12:43:55.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: grep '^pool.*['\'']test['\'']' 2026-02-19T12:43:55.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: awk '{ print $2 }' 2026-02-19T12:43:55.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: poolid=1 2026-02-19T12:43:55.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:65: TEST_recovery_scrub_1: ceph pg dump pgs 2026-02-19T12:43:56.041 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:43:56.041 INFO:tasks.workunit.client.0.trial144.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T12:43:52.114938+0000 0'0 19:9 [1] 1 [1] 1 0'0 2026-02-19T12:43:51.808651+0000 0'0 2026-02-19T12:43:51.808651+0000 0 0 periodic scrub scheduled @ 2026-02-20T20:49:54.844026+0000 0 0 2026-02-19T12:43:56.041 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:43:56.042 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:43:56.042 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:43:56.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:67: TEST_recovery_scrub_1: dd if=/dev/urandom of=testdata.26613 bs=1M count=50 2026-02-19T12:43:56.176 INFO:tasks.workunit.client.0.trial144.stderr:50+0 records in 2026-02-19T12:43:56.177 INFO:tasks.workunit.client.0.trial144.stderr:50+0 records out 2026-02-19T12:43:56.177 INFO:tasks.workunit.client.0.trial144.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.12353 s, 424 MB/s 2026-02-19T12:43:56.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: seq 1 100 2026-02-19T12:43:56.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:56.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj1 testdata.26613 2026-02-19T12:43:56.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:56.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj2 testdata.26613 2026-02-19T12:43:56.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:56.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj3 testdata.26613 2026-02-19T12:43:56.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:56.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj4 testdata.26613 2026-02-19T12:43:56.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:56.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj5 testdata.26613 2026-02-19T12:43:56.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:56.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj6 testdata.26613 2026-02-19T12:43:56.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:56.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj7 testdata.26613 2026-02-19T12:43:56.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:56.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj8 testdata.26613 2026-02-19T12:43:56.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:56.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj9 testdata.26613 2026-02-19T12:43:56.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:56.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj10 testdata.26613 2026-02-19T12:43:56.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:56.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj11 testdata.26613 2026-02-19T12:43:57.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:57.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj12 testdata.26613 2026-02-19T12:43:57.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:57.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj13 testdata.26613 2026-02-19T12:43:57.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:57.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj14 testdata.26613 2026-02-19T12:43:57.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:57.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj15 testdata.26613 2026-02-19T12:43:57.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:57.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj16 testdata.26613 2026-02-19T12:43:57.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:57.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj17 testdata.26613 2026-02-19T12:43:57.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:57.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj18 testdata.26613 2026-02-19T12:43:57.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:57.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj19 testdata.26613 2026-02-19T12:43:57.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:57.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj20 testdata.26613 2026-02-19T12:43:57.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:57.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj21 testdata.26613 2026-02-19T12:43:57.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:57.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj22 testdata.26613 2026-02-19T12:43:57.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:57.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj23 testdata.26613 2026-02-19T12:43:58.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:58.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj24 testdata.26613 2026-02-19T12:43:58.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:58.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj25 testdata.26613 2026-02-19T12:43:58.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:58.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj26 testdata.26613 2026-02-19T12:43:58.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:58.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj27 testdata.26613 2026-02-19T12:43:58.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:58.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj28 testdata.26613 2026-02-19T12:43:58.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:58.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj29 testdata.26613 2026-02-19T12:43:58.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:58.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj30 testdata.26613 2026-02-19T12:43:58.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:58.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj31 testdata.26613 2026-02-19T12:43:58.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:58.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj32 testdata.26613 2026-02-19T12:43:58.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:58.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj33 testdata.26613 2026-02-19T12:43:58.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:58.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj34 testdata.26613 2026-02-19T12:43:58.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:58.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj35 testdata.26613 2026-02-19T12:43:58.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:58.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj36 testdata.26613 2026-02-19T12:43:59.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:59.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj37 testdata.26613 2026-02-19T12:43:59.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:59.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj38 testdata.26613 2026-02-19T12:43:59.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:59.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj39 testdata.26613 2026-02-19T12:43:59.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:59.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj40 testdata.26613 2026-02-19T12:43:59.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:59.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj41 testdata.26613 2026-02-19T12:43:59.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:59.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj42 testdata.26613 2026-02-19T12:43:59.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:59.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj43 testdata.26613 2026-02-19T12:43:59.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:59.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj44 testdata.26613 2026-02-19T12:43:59.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:59.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj45 testdata.26613 2026-02-19T12:43:59.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:59.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj46 testdata.26613 2026-02-19T12:43:59.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:59.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj47 testdata.26613 2026-02-19T12:43:59.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:43:59.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj48 testdata.26613 2026-02-19T12:44:00.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:00.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj49 testdata.26613 2026-02-19T12:44:00.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:00.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj50 testdata.26613 2026-02-19T12:44:00.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:00.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj51 testdata.26613 2026-02-19T12:44:00.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:00.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj52 testdata.26613 2026-02-19T12:44:00.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:00.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj53 testdata.26613 2026-02-19T12:44:00.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:00.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj54 testdata.26613 2026-02-19T12:44:00.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:00.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj55 testdata.26613 2026-02-19T12:44:00.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:00.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj56 testdata.26613 2026-02-19T12:44:00.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:00.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj57 testdata.26613 2026-02-19T12:44:00.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:00.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj58 testdata.26613 2026-02-19T12:44:00.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:00.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj59 testdata.26613 2026-02-19T12:44:00.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:00.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj60 testdata.26613 2026-02-19T12:44:01.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:01.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj61 testdata.26613 2026-02-19T12:44:01.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:01.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj62 testdata.26613 2026-02-19T12:44:01.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:01.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj63 testdata.26613 2026-02-19T12:44:01.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:01.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj64 testdata.26613 2026-02-19T12:44:01.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:01.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj65 testdata.26613 2026-02-19T12:44:01.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:01.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj66 testdata.26613 2026-02-19T12:44:01.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:01.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj67 testdata.26613 2026-02-19T12:44:01.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:01.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj68 testdata.26613 2026-02-19T12:44:01.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:01.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj69 testdata.26613 2026-02-19T12:44:01.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:01.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj70 testdata.26613 2026-02-19T12:44:01.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:01.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj71 testdata.26613 2026-02-19T12:44:01.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:01.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj72 testdata.26613 2026-02-19T12:44:01.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:01.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj73 testdata.26613 2026-02-19T12:44:02.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:02.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj74 testdata.26613 2026-02-19T12:44:02.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:02.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj75 testdata.26613 2026-02-19T12:44:02.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:02.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj76 testdata.26613 2026-02-19T12:44:02.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:02.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj77 testdata.26613 2026-02-19T12:44:02.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:02.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj78 testdata.26613 2026-02-19T12:44:02.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:02.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj79 testdata.26613 2026-02-19T12:44:02.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:02.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj80 testdata.26613 2026-02-19T12:44:02.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:02.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj81 testdata.26613 2026-02-19T12:44:02.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:02.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj82 testdata.26613 2026-02-19T12:44:02.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:02.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj83 testdata.26613 2026-02-19T12:44:02.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:02.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj84 testdata.26613 2026-02-19T12:44:03.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:03.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj85 testdata.26613 2026-02-19T12:44:03.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:03.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj86 testdata.26613 2026-02-19T12:44:03.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:03.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj87 testdata.26613 2026-02-19T12:44:03.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:03.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj88 testdata.26613 2026-02-19T12:44:03.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:03.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj89 testdata.26613 2026-02-19T12:44:03.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:03.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj90 testdata.26613 2026-02-19T12:44:03.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:03.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj91 testdata.26613 2026-02-19T12:44:03.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:03.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj92 testdata.26613 2026-02-19T12:44:03.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:03.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj93 testdata.26613 2026-02-19T12:44:03.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:03.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj94 testdata.26613 2026-02-19T12:44:03.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:03.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj95 testdata.26613 2026-02-19T12:44:03.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:03.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj96 testdata.26613 2026-02-19T12:44:04.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:04.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj97 testdata.26613 2026-02-19T12:44:04.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:04.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj98 testdata.26613 2026-02-19T12:44:04.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:04.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj99 testdata.26613 2026-02-19T12:44:04.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-19T12:44:04.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj100 testdata.26613 2026-02-19T12:44:04.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:72: TEST_recovery_scrub_1: rm -f testdata.26613 2026-02-19T12:44:04.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:74: TEST_recovery_scrub_1: ceph osd pool set test size 4 2026-02-19T12:44:04.702 INFO:tasks.workunit.client.0.trial144.stderr:set pool 1 size to 4 2026-02-19T12:44:04.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:77: TEST_recovery_scrub_1: set -o pipefail 2026-02-19T12:44:04.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:78: TEST_recovery_scrub_1: count=0 2026-02-19T12:44:04.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2026-02-19T12:44:04.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2026-02-19T12:44:04.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2026-02-19T12:44:04.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2026-02-19T12:44:04.862 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:44:04.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:86: TEST_recovery_scrub_1: sleep 2 2026-02-19T12:44:06.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:87: TEST_recovery_scrub_1: test 0 -eq 10 2026-02-19T12:44:06.873 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: expr 0 + 1 2026-02-19T12:44:06.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: count=1 2026-02-19T12:44:06.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2026-02-19T12:44:06.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2026-02-19T12:44:06.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2026-02-19T12:44:06.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2026-02-19T12:44:07.037 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:44:07.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:86: TEST_recovery_scrub_1: sleep 2 2026-02-19T12:44:09.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:87: TEST_recovery_scrub_1: test 1 -eq 10 2026-02-19T12:44:09.051 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: expr 1 + 1 2026-02-19T12:44:09.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: count=2 2026-02-19T12:44:09.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2026-02-19T12:44:09.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2026-02-19T12:44:09.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2026-02-19T12:44:09.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2026-02-19T12:44:09.219 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:44:09.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:84: TEST_recovery_scrub_1: break 2026-02-19T12:44:09.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:94: TEST_recovery_scrub_1: set +o pipefail 2026-02-19T12:44:09.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:95: TEST_recovery_scrub_1: ceph pg dump pgs 2026-02-19T12:44:09.377 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:44:09.377 INFO:tasks.workunit.client.0.trial144.stdout:1.0 100 0 279 0 0 5242880000 0 0 1300 0 1300 active+recovering+undersized+degraded+remapped 2026-02-19T12:44:05.626018+0000 20'1300 23:1351 [1,0,2,3] 1 [1,0] 1 0'0 2026-02-19T12:43:51.808651+0000 0'0 2026-02-19T12:43:51.808651+0000 0 0 no scrub is scheduled 0 0 2026-02-19T12:44:09.377 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:44:09.378 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:44:09.378 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:44:09.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:97: TEST_recovery_scrub_1: sleep 10 2026-02-19T12:44:19.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:99: TEST_recovery_scrub_1: kill_daemons td/osd-recovery-scrub 2026-02-19T12:44:19.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:44:19.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:44:19.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:44:19.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:44:19.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:44:24.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:44:24.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:101: TEST_recovery_scrub_1: declare -a err_strings 2026-02-19T12:44:24.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:102: TEST_recovery_scrub_1: err_strings[0]='recovery in progress.*scrubs' 2026-02-19T12:44:24.711 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: expr 4 - 1 2026-02-19T12:44:24.713 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: seq 0 3 2026-02-19T12:44:24.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T12:44:24.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'recovery in progress.*scrubs' td/osd-recovery-scrub/osd.0.log 2026-02-19T12:44:24.719 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:05.703+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.719 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:06.731+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.719 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:07.699+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.719 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:08.679+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.719 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:09.683+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.719 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:10.690+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.720 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:11.694+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.720 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:12.706+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.720 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:13.662+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.720 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:14.706+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.720 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:15.754+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.720 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:16.778+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.720 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:17.782+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.720 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:18.830+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.720 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:19.822+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.720 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:20.802+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.720 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:21.754+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.721 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:22.770+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.721 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:23.766+0000 7f62f83eb640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T12:44:24.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'recovery in progress.*scrubs' td/osd-recovery-scrub/osd.1.log 2026-02-19T12:44:24.733 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:06.103+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.733 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:07.115+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.733 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:08.163+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.733 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:09.151+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.733 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:10.147+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.734 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:11.146+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.734 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:12.138+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.734 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:13.186+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.734 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:14.206+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.734 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:15.162+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.734 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:16.166+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.734 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:17.122+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.734 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:18.122+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.734 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:19.170+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.734 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:20.130+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.734 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:21.166+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.734 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:22.158+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.734 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:23.166+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.735 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:24.198+0000 7f5b0fb52640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T12:44:24.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'recovery in progress.*scrubs' td/osd-recovery-scrub/osd.2.log 2026-02-19T12:44:24.738 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:06.651+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.738 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:07.651+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.738 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:08.659+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.738 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:09.703+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.738 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:10.662+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.738 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:11.662+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.738 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:12.658+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.738 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:13.626+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.739 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:14.622+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.739 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:15.602+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.739 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:16.574+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.739 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:17.594+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.739 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:18.554+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.739 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:19.562+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.739 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:20.602+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.739 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:21.602+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.739 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:22.570+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.739 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:23.578+0000 7f48dad64640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T12:44:24.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'recovery in progress.*scrubs' td/osd-recovery-scrub/osd.3.log 2026-02-19T12:44:24.743 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:06.131+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.743 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:07.179+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.743 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:08.135+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.743 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:09.151+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.743 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:10.139+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.743 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:11.182+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.744 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:12.230+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.744 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:13.190+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.744 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:14.154+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.744 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:15.146+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.744 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:16.170+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.744 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:17.206+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.744 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:18.242+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.744 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:19.254+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.744 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:20.262+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.744 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:21.282+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.744 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:22.234+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.744 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:23.234+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.744 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:44:24.274+0000 7f4419359640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Operator-initiated scrubs only. 2026-02-19T12:44:24.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:108: TEST_recovery_scrub_1: for err_string in "${err_strings[@]}" 2026-02-19T12:44:24.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:110: TEST_recovery_scrub_1: found=false 2026-02-19T12:44:24.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:111: TEST_recovery_scrub_1: count=0 2026-02-19T12:44:24.746 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: expr 4 - 1 2026-02-19T12:44:24.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: seq 0 3 2026-02-19T12:44:24.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T12:44:24.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'recovery in progress.*scrubs' td/osd-recovery-scrub/osd.0.log 2026-02-19T12:44:24.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2026-02-19T12:44:24.748 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 0 + 1 2026-02-19T12:44:24.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=1 2026-02-19T12:44:24.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T12:44:24.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'recovery in progress.*scrubs' td/osd-recovery-scrub/osd.1.log 2026-02-19T12:44:24.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2026-02-19T12:44:24.759 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 1 + 1 2026-02-19T12:44:24.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=2 2026-02-19T12:44:24.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T12:44:24.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'recovery in progress.*scrubs' td/osd-recovery-scrub/osd.2.log 2026-02-19T12:44:24.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2026-02-19T12:44:24.762 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 2 + 1 2026-02-19T12:44:24.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=3 2026-02-19T12:44:24.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T12:44:24.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'recovery in progress.*scrubs' td/osd-recovery-scrub/osd.3.log 2026-02-19T12:44:24.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2026-02-19T12:44:24.765 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 3 + 1 2026-02-19T12:44:24.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=4 2026-02-19T12:44:24.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:120: TEST_recovery_scrub_1: '[' true = false ']' 2026-02-19T12:44:24.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:124: TEST_recovery_scrub_1: '[' 4 -eq 4 ']' 2026-02-19T12:44:24.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:127: TEST_recovery_scrub_1: teardown td/osd-recovery-scrub 2026-02-19T12:44:24.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-recovery-scrub 2026-02-19T12:44:24.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:44:24.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-recovery-scrub KILL 2026-02-19T12:44:24.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:44:24.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:44:24.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:44:24.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:44:24.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:44:24.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:44:24.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:44:24.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:44:24.774 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:44:24.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:44:24.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:44:24.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:44:24.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:44:24.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:44:24.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:44:24.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:44:24.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:44:24.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:44:24.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-recovery-scrub 2026-02-19T12:44:24.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:44:24.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:24.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.26613 2026-02-19T12:44:24.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.26613 2026-02-19T12:44:24.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:44:24.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:44:24.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:129: TEST_recovery_scrub_1: '[' 0 '!=' 0 ']' 2026-02-19T12:44:24.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:135: TEST_recovery_scrub_1: echo 'TEST PASSED' 2026-02-19T12:44:24.826 INFO:tasks.workunit.client.0.trial144.stdout:TEST PASSED 2026-02-19T12:44:24.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:136: TEST_recovery_scrub_1: return 0 2026-02-19T12:44:24.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-19T12:44:24.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/osd-recovery-scrub 0 2026-02-19T12:44:24.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-recovery-scrub 2026-02-19T12:44:24.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-19T12:44:24.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-recovery-scrub KILL 2026-02-19T12:44:24.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:44:24.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:44:24.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:44:24.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:44:24.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:44:24.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:44:24.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:44:24.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:44:24.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:44:24.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:44:24.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:44:24.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:44:24.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:44:24.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:44:24.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:44:24.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:44:24.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:44:24.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-19T12:44:24.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-recovery-scrub 2026-02-19T12:44:24.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:44:24.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:24.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.26613 2026-02-19T12:44:24.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.26613 2026-02-19T12:44:24.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:44:24.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:44:24.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-19T12:44:24.838 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-19T12:44:24.839 DEBUG:teuthology.orchestra.run.trial144:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-19T12:44:24.893 INFO:tasks.workunit:Running workunit scrub/osd-scrub-dump.sh... 2026-02-19T12:44:24.894 DEBUG:teuthology.orchestra.run.trial144:workunit test scrub/osd-scrub-dump.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=624b9a7bcb2064ce17206482df6a1da007ff7675 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh 2026-02-19T12:44:24.946 INFO:tasks.workunit.client.0.trial144.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-19T12:44:24.949 INFO:tasks.workunit.client.0.trial144.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-19T12:44:24.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-19T12:44:24.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-19T12:44:24.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-19T12:44:24.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-19T12:44:24.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-19T12:44:24.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-19T12:44:24.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-19T12:44:24.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-19T12:44:24.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/osd-scrub-dump 2026-02-19T12:44:24.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:30: run: echo 'This test is disabled' 2026-02-19T12:44:24.951 INFO:tasks.workunit.client.0.trial144.stdout:This test is disabled 2026-02-19T12:44:24.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:31: run: return 0 2026-02-19T12:44:24.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-19T12:44:24.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/osd-scrub-dump 0 2026-02-19T12:44:24.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-dump 2026-02-19T12:44:24.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-19T12:44:24.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-dump KILL 2026-02-19T12:44:24.952 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:44:24.952 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:44:24.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:44:24.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:44:24.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:44:24.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:44:24.953 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:44:24.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:44:24.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:44:24.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:44:24.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:44:24.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:44:24.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:44:24.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:44:24.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:44:24.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:44:24.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:44:24.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-19T12:44:24.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-dump 2026-02-19T12:44:24.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:44:24.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:24.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35794 2026-02-19T12:44:24.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35794 2026-02-19T12:44:24.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:44:24.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:44:24.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-19T12:44:24.962 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-19T12:44:24.963 DEBUG:teuthology.orchestra.run.trial144:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-19T12:44:25.013 INFO:tasks.workunit:Running workunit scrub/osd-scrub-repair.sh... 2026-02-19T12:44:25.013 DEBUG:teuthology.orchestra.run.trial144:workunit test scrub/osd-scrub-repair.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=624b9a7bcb2064ce17206482df6a1da007ff7675 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh 2026-02-19T12:44:25.064 INFO:tasks.workunit.client.0.trial144.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh 2026-02-19T12:44:25.065 INFO:tasks.workunit.client.0.trial144.stderr:++ TIMEOUT=300 2026-02-19T12:44:25.065 INFO:tasks.workunit.client.0.trial144.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2026-02-19T12:44:25.065 INFO:tasks.workunit.client.0.trial144.stderr:++ MAX_TIMEOUT=15 2026-02-19T12:44:25.065 INFO:tasks.workunit.client.0.trial144.stderr:++ PG_NUM=4 2026-02-19T12:44:25.065 INFO:tasks.workunit.client.0.trial144.stderr:++ TMPDIR=/tmp 2026-02-19T12:44:25.065 INFO:tasks.workunit.client.0.trial144.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2026-02-19T12:44:25.065 INFO:tasks.workunit.client.0.trial144.stderr:++ TESTDIR=/home/ubuntu/cephtest 2026-02-19T12:44:25.065 INFO:tasks.workunit.client.0.trial144.stderr:+++ uname 2026-02-19T12:44:25.066 INFO:tasks.workunit.client.0.trial144.stderr:++ '[' Linux = FreeBSD ']' 2026-02-19T12:44:25.066 INFO:tasks.workunit.client.0.trial144.stderr:++ SED=sed 2026-02-19T12:44:25.066 INFO:tasks.workunit.client.0.trial144.stderr:++ AWK=awk 2026-02-19T12:44:25.067 INFO:tasks.workunit.client.0.trial144.stderr:+++ stty -a 2026-02-19T12:44:25.067 INFO:tasks.workunit.client.0.trial144.stderr:+++ head -1 2026-02-19T12:44:25.067 INFO:tasks.workunit.client.0.trial144.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2026-02-19T12:44:25.067 INFO:tasks.workunit.client.0.trial144.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-19T12:44:25.068 INFO:tasks.workunit.client.0.trial144.stderr:++ termwidth= 2026-02-19T12:44:25.068 INFO:tasks.workunit.client.0.trial144.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2026-02-19T12:44:25.068 INFO:tasks.workunit.client.0.trial144.stderr:++ DIFFCOLOPTS='-y ' 2026-02-19T12:44:25.068 INFO:tasks.workunit.client.0.trial144.stderr:++ KERNCORE=kernel.core_pattern 2026-02-19T12:44:25.068 INFO:tasks.workunit.client.0.trial144.stderr:++ EXTRA_OPTS= 2026-02-19T12:44:25.069 INFO:tasks.workunit.client.0.trial144.stderr:++ test '' = TESTS 2026-02-19T12:44:25.069 INFO:tasks.workunit.client.0.trial144.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh 2026-02-19T12:44:25.070 INFO:tasks.workunit.client.0.trial144.stderr:++ uname 2026-02-19T12:44:25.071 INFO:tasks.workunit.client.0.trial144.stderr:+ '[' Linux = FreeBSD ']' 2026-02-19T12:44:25.071 INFO:tasks.workunit.client.0.trial144.stderr:+ use_ec_overwrite=true 2026-02-19T12:44:25.071 INFO:tasks.workunit.client.0.trial144.stderr:+ getjson=no 2026-02-19T12:44:25.071 INFO:tasks.workunit.client.0.trial144.stderr:+ jqfilter='def walk(f): 2026-02-19T12:44:25.071 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T12:44:25.071 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T12:44:25.072 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T12:44:25.072 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T12:44:25.072 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T12:44:25.072 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T12:44:25.072 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T12:44:25.072 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T12:44:25.072 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T12:44:25.072 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T12:44:25.072 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T12:44:25.072 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-19T12:44:25.072 INFO:tasks.workunit.client.0.trial144.stderr:+ sortkeys='import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T12:44:25.077 INFO:tasks.workunit.client.0.trial144.stderr:+ main osd-scrub-repair 2026-02-19T12:44:25.077 INFO:tasks.workunit.client.0.trial144.stderr:+ local dir=td/osd-scrub-repair 2026-02-19T12:44:25.077 INFO:tasks.workunit.client.0.trial144.stderr:+ shift 2026-02-19T12:44:25.077 INFO:tasks.workunit.client.0.trial144.stderr:+ shopt -s -o xtrace 2026-02-19T12:44:25.077 INFO:tasks.workunit.client.0.trial144.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-19T12:44:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-19T12:44:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-19T12:44:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-19T12:44:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-19T12:44:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-19T12:44:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-19T12:44:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-19T12:44:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-19T12:44:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/osd-scrub-repair 2026-02-19T12:44:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:52: run: local dir=td/osd-scrub-repair 2026-02-19T12:44:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:53: run: shift 2026-02-19T12:44:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:55: run: export CEPH_MON=127.0.0.1:7107 2026-02-19T12:44:25.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:55: run: CEPH_MON=127.0.0.1:7107 2026-02-19T12:44:25.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:56: run: export CEPH_ARGS 2026-02-19T12:44:25.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:57: run: uuidgen 2026-02-19T12:44:25.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:57: run: CEPH_ARGS+='--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none ' 2026-02-19T12:44:25.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:58: run: CEPH_ARGS+='--mon-host=127.0.0.1:7107 ' 2026-02-19T12:44:25.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:59: run: CEPH_ARGS+='--osd-skip-data-digest=false ' 2026-02-19T12:44:25.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:61: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-19T12:44:25.080 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:62: run: set 2026-02-19T12:44:25.080 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:62: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-19T12:44:25.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:62: run: local 'funcs=TEST_allow_oper_initiated_scrub_during_recovery 2026-02-19T12:44:25.083 INFO:tasks.workunit.client.0.trial144.stderr:TEST_allow_repair_during_recovery 2026-02-19T12:44:25.083 INFO:tasks.workunit.client.0.trial144.stderr:TEST_auto_repair_bluestore_basic 2026-02-19T12:44:25.083 INFO:tasks.workunit.client.0.trial144.stderr:TEST_auto_repair_bluestore_failed 2026-02-19T12:44:25.083 INFO:tasks.workunit.client.0.trial144.stderr:TEST_auto_repair_bluestore_failed_norecov 2026-02-19T12:44:25.083 INFO:tasks.workunit.client.0.trial144.stderr:TEST_auto_repair_bluestore_scrub 2026-02-19T12:44:25.083 INFO:tasks.workunit.client.0.trial144.stderr:TEST_auto_repair_bluestore_tag 2026-02-19T12:44:25.084 INFO:tasks.workunit.client.0.trial144.stderr:TEST_auto_repair_erasure_coded_appends 2026-02-19T12:44:25.084 INFO:tasks.workunit.client.0.trial144.stderr:TEST_auto_repair_erasure_coded_overwrites 2026-02-19T12:44:25.084 INFO:tasks.workunit.client.0.trial144.stderr:TEST_corrupt_and_repair_jerasure_appends 2026-02-19T12:44:25.084 INFO:tasks.workunit.client.0.trial144.stderr:TEST_corrupt_and_repair_jerasure_overwrites 2026-02-19T12:44:25.084 INFO:tasks.workunit.client.0.trial144.stderr:TEST_corrupt_and_repair_lrc_appends 2026-02-19T12:44:25.084 INFO:tasks.workunit.client.0.trial144.stderr:TEST_corrupt_and_repair_lrc_overwrites 2026-02-19T12:44:25.084 INFO:tasks.workunit.client.0.trial144.stderr:TEST_corrupt_and_repair_replicated 2026-02-19T12:44:25.084 INFO:tasks.workunit.client.0.trial144.stderr:TEST_corrupt_scrub_erasure_appends 2026-02-19T12:44:25.084 INFO:tasks.workunit.client.0.trial144.stderr:TEST_corrupt_scrub_erasure_overwrites 2026-02-19T12:44:25.084 INFO:tasks.workunit.client.0.trial144.stderr:TEST_corrupt_scrub_replicated 2026-02-19T12:44:25.084 INFO:tasks.workunit.client.0.trial144.stderr:TEST_corrupt_snapset_scrub_rep 2026-02-19T12:44:25.084 INFO:tasks.workunit.client.0.trial144.stderr:TEST_dual_store_replicated_cluster 2026-02-19T12:44:25.085 INFO:tasks.workunit.client.0.trial144.stderr:TEST_list_missing_erasure_coded_appends 2026-02-19T12:44:25.085 INFO:tasks.workunit.client.0.trial144.stderr:TEST_list_missing_erasure_coded_overwrites 2026-02-19T12:44:25.085 INFO:tasks.workunit.client.0.trial144.stderr:TEST_periodic_scrub_replicated 2026-02-19T12:44:25.085 INFO:tasks.workunit.client.0.trial144.stderr:TEST_repair_stats 2026-02-19T12:44:25.085 INFO:tasks.workunit.client.0.trial144.stderr:TEST_repair_stats_ec 2026-02-19T12:44:25.085 INFO:tasks.workunit.client.0.trial144.stderr:TEST_request_scrub_priority 2026-02-19T12:44:25.085 INFO:tasks.workunit.client.0.trial144.stderr:TEST_scrub_warning 2026-02-19T12:44:25.085 INFO:tasks.workunit.client.0.trial144.stderr:TEST_skip_non_repair_during_recovery 2026-02-19T12:44:25.085 INFO:tasks.workunit.client.0.trial144.stderr:TEST_unfound_erasure_coded_appends 2026-02-19T12:44:25.085 INFO:tasks.workunit.client.0.trial144.stderr:TEST_unfound_erasure_coded_overwrites' 2026-02-19T12:44:25.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T12:44:25.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T12:44:25.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T12:44:25.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T12:44:25.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:44:25.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:44:25.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:44:25.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:44:25.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:44:25.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:44:25.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:44:25.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:44:25.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:44:25.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:44:25.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:44:25.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:44:25.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:44:25.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:44:25.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:44:25.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:44:25.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:44:25.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:44:25.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:44:25.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:44:25.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:44:25.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:44:25.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:44:25.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:25.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:44:25.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:44:25.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:44:25.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:44:25.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T12:44:25.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:44:25.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:25.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:44:25.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T12:44:25.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:44:25.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-19T12:44:25.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-19T12:44:25.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:44:25.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T12:44:25.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_allow_oper_initiated_scrub_during_recovery td/osd-scrub-repair 2026-02-19T12:44:25.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:115: TEST_allow_oper_initiated_scrub_during_recovery: local dir=td/osd-scrub-repair 2026-02-19T12:44:25.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:116: TEST_allow_oper_initiated_scrub_during_recovery: cluster_conf=(['osds_num']='2' ['pgs_in_pool']='4' ['pool_name']='nopool' ['pool_default_size']='2' ['extras']='--osd_scrub_during_recovery=false --osd_debug_pretend_recovery_active=true') 2026-02-19T12:44:25.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:116: TEST_allow_oper_initiated_scrub_during_recovery: local -A cluster_conf 2026-02-19T12:44:25.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:125: TEST_allow_oper_initiated_scrub_during_recovery: standard_scrub_cluster td/osd-scrub-repair cluster_conf 2026-02-19T12:44:25.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:229: standard_scrub_cluster: local dir=td/osd-scrub-repair 2026-02-19T12:44:25.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:230: standard_scrub_cluster: local -n args=cluster_conf 2026-02-19T12:44:25.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:232: standard_scrub_cluster: local OSDS=2 2026-02-19T12:44:25.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:233: standard_scrub_cluster: local pg_num=4 2026-02-19T12:44:25.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:235: standard_scrub_cluster: local poolname=nopool 2026-02-19T12:44:25.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:236: standard_scrub_cluster: args['pool_name']=nopool 2026-02-19T12:44:25.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:238: standard_scrub_cluster: local pool_default_size=2 2026-02-19T12:44:25.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:239: standard_scrub_cluster: args['pool_default_size']=2 2026-02-19T12:44:25.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:241: standard_scrub_cluster: local 'extra_pars=--osd_scrub_during_recovery=false --osd_debug_pretend_recovery_active=true' 2026-02-19T12:44:25.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:242: standard_scrub_cluster: local debug_msg=dbg 2026-02-19T12:44:25.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:245: standard_scrub_cluster: local saved_echo_flag=x 2026-02-19T12:44:25.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:246: standard_scrub_cluster: set +x 2026-02-19T12:44:25.386 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 ad919b88-88ff-4084-803b-337957835710 2026-02-19T12:44:25.487 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:44:25.516 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:25.510+0000 7fd01479ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:25.517 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:25.514+0000 7fd01479ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:25.518 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:25.514+0000 7fd01479ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:25.519 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:25.514+0000 7fd01479ba40 -1 bdev(0x561dd856cc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:44:25.519 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:25.514+0000 7fd01479ba40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T12:44:26.244 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:44:26.446 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:44:26.479 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:26.474+0000 7f3c93014a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:26.480 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:26.474+0000 7f3c93014a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:26.482 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:26.478+0000 7f3c93014a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:26.608 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:44:26.866 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:26.862+0000 7f3c93014a40 -1 Falling back to public interface 2026-02-19T12:44:27.483 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:27.478+0000 7f3c93014a40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:44:27.770 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:44:28.939 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:44:29.098 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2725374924,v1:127.0.0.1:6803/2725374924] [v2:127.0.0.1:6804/2725374924,v1:127.0.0.1:6805/2725374924] exists,up ad919b88-88ff-4084-803b-337957835710 2026-02-19T12:44:29.103 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 bea9a9a9-ba2c-4943-a523-52ca1ab27209 2026-02-19T12:44:29.278 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:44:29.311 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:29.306+0000 7f4478deca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:29.312 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:29.306+0000 7f4478deca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:29.313 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:29.310+0000 7f4478deca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:29.314 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:29.310+0000 7f4478deca40 -1 bdev(0x558c3be32c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:44:29.314 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:29.310+0000 7f4478deca40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T12:44:29.932 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:44:30.135 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:44:30.155 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:30.150+0000 7f06b8892a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:30.156 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:30.150+0000 7f06b8892a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:30.157 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:30.154+0000 7f06b8892a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:30.296 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:44:30.563 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:30.558+0000 7f06b8892a40 -1 Falling back to public interface 2026-02-19T12:44:31.179 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:31.174+0000 7f06b8892a40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:44:31.458 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:44:32.618 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:44:32.772 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1511065141,v1:127.0.0.1:6811/1511065141] [v2:127.0.0.1:6812/1511065141,v1:127.0.0.1:6813/1511065141] exists,up bea9a9a9-ba2c-4943-a523-52ca1ab27209 2026-02-19T12:44:33.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:126: TEST_allow_oper_initiated_scrub_during_recovery: local poolname=rbd 2026-02-19T12:44:33.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:127: TEST_allow_oper_initiated_scrub_during_recovery: create_rbd_pool 2026-02-19T12:44:33.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T12:44:33.922 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T12:44:33.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T12:44:33.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T12:44:34.109 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T12:44:34.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:44:35.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T12:44:35.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:128: TEST_allow_oper_initiated_scrub_during_recovery: wait_for_clean 2026-02-19T12:44:35.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:44:35.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:44:35.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:44:35.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:44:35.402 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:44:35.402 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:44:35.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:44:35.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:44:35.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:44:35.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:44:35.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:44:35.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:44:35.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:44:35.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:44:35.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:44:35.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:44:35.667 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T12:44:35.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:44:35.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:44:35.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:44:35.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836489 2026-02-19T12:44:35.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836489 2026-02-19T12:44:35.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489' 2026-02-19T12:44:35.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:44:35.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:44:35.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705669 2026-02-19T12:44:35.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705669 2026-02-19T12:44:35.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-38654705669' 2026-02-19T12:44:35.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:44:35.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836489 2026-02-19T12:44:35.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:44:35.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:44:35.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836489 2026-02-19T12:44:35.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:44:35.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836489 2026-02-19T12:44:35.815 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836489 2026-02-19T12:44:35.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-19T12:44:35.816 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:44:35.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-19T12:44:35.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:44:35.984 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705669 2026-02-19T12:44:35.984 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:44:35.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:44:35.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705669 2026-02-19T12:44:35.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:44:35.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705669 2026-02-19T12:44:35.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-19T12:44:35.988 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705669 2026-02-19T12:44:35.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:44:36.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-19T12:44:36.156 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:44:36.156 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:44:36.156 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:44:36.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:44:36.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:44:36.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:44:36.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:44:36.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:44:36.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:44:36.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:44:36.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:44:36.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:44:36.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:44:36.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:44:36.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:44:36.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:44:36.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:44:36.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:44:36.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:130: TEST_allow_oper_initiated_scrub_during_recovery: add_something td/osd-scrub-repair rbd 2026-02-19T12:44:36.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T12:44:36.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=rbd 2026-02-19T12:44:36.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-19T12:44:36.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T12:44:36.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T12:44:36.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T12:44:36.906 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T12:44:36.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T12:44:37.110 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T12:44:37.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T12:44:37.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T12:44:37.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool rbd put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-19T12:44:37.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:131: TEST_allow_oper_initiated_scrub_during_recovery: get_not_primary rbd SOMETHING 2026-02-19T12:44:37.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=rbd 2026-02-19T12:44:37.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:44:37.146 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary rbd SOMETHING 2026-02-19T12:44:37.146 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-19T12:44:37.146 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:44:37.146 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd SOMETHING 2026-02-19T12:44:37.146 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:44:37.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:44:37.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map rbd SOMETHING 2026-02-19T12:44:37.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:44:37.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:131: TEST_allow_oper_initiated_scrub_during_recovery: oper_scrub_and_schedule td/osd-scrub-repair rbd 0 2026-02-19T12:44:37.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:180: oper_scrub_and_schedule: local dir=td/osd-scrub-repair 2026-02-19T12:44:37.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:181: oper_scrub_and_schedule: local poolname=rbd 2026-02-19T12:44:37.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:182: oper_scrub_and_schedule: local osd=0 2026-02-19T12:44:37.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: oper_scrub_and_schedule: get_pg rbd SOMETHING 2026-02-19T12:44:37.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=rbd 2026-02-19T12:44:37.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:44:37.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map rbd SOMETHING 2026-02-19T12:44:37.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:44:37.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: oper_scrub_and_schedule: local pg=1.3 2026-02-19T12:44:37.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: oper_scrub_and_schedule: get_last_scrub_stamp 1.3 2026-02-19T12:44:37.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:44:37.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:44:37.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:44:37.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:44:37.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: oper_scrub_and_schedule: local last_scrub=2026-02-19T12:44:34.105763+0000 2026-02-19T12:44:37.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:189: oper_scrub_and_schedule: ceph tell 1.3 scrub 2026-02-19T12:44:37.845 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T12:44:37.845 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T12:44:37.845 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T12:44:37.845 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T12:44:37.845 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:44:37.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: oper_scrub_and_schedule: sleep 3 2026-02-19T12:44:40.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: oper_scrub_and_schedule: (( i=0 )) 2026-02-19T12:44:40.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: oper_scrub_and_schedule: (( i < 3 )) 2026-02-19T12:44:40.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: oper_scrub_and_schedule: get_last_scrub_stamp 1.3 2026-02-19T12:44:40.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:44:40.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:44:40.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:44:40.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:44:41.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: oper_scrub_and_schedule: test 2026-02-19T12:44:38.284807+0000 '>' 2026-02-19T12:44:34.105763+0000 2026-02-19T12:44:41.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: oper_scrub_and_schedule: break 2026-02-19T12:44:41.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: oper_scrub_and_schedule: objectstore_tool td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-19T12:44:41.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:44:41.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:44:41.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:44:41.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:44:41.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-19T12:44:41.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:44:41.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:44:41.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:44:41.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:44:41.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:44:41.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:44:41.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:44:41.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:44:41.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:44:41.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:44:41.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:44:41.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-19T12:44:41.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:44:41.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:44:41.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:44:41.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:44:41.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:44:41.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING list-attrs 2026-02-19T12:44:41.407 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:44:41.407 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:44:41.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T12:44:41.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:44:41.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:44:41.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:44:41.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:44:41.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:44:41.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:44:41.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:44:41.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:44:41.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:44:41.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:44:41.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:44:41.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:44:41.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:44:41.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:44:41.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:44:41.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:44:41.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:44:41.520 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:44:41.520 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:41.520 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:44:41.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:44:41.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:44:41.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:44:41.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:44:41.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:44:41.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:44:41.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:44:41.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:44:41.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:44:41.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:44:41.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:44:41.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:44:41.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:44:41.523 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:44:41.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:44:41.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:44:41.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:44:41.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:44:41.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:44:41.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:44:41.530 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:41.526+0000 7fbb02aaea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:41.531 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:41.526+0000 7fbb02aaea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:41.532 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:41.526+0000 7fbb02aaea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:41.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:44:41.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:44:41.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:44:41.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:44:41.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:44:41.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:44:41.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:44:41.674 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:44:41.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:44:41.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:44:41.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:44:41.895 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:41.890+0000 7fbb02aaea40 -1 Falling back to public interface 2026-02-19T12:44:42.565 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:42.562+0000 7fbb02aaea40 -1 osd.0 18 log_to_monitors true 2026-02-19T12:44:42.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:44:42.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:44:42.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:44:42.835 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:44:42.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:44:42.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:44:42.991 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 22 up_thru 22 down_at 19 last_clean_interval [5,18) [v2:127.0.0.1:6802/1498695321,v1:127.0.0.1:6803/1498695321] [v2:127.0.0.1:6804/1498695321,v1:127.0.0.1:6805/1498695321] exists,up ad919b88-88ff-4084-803b-337957835710 2026-02-19T12:44:42.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:44:42.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:44:42.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:44:42.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:44:42.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:44:42.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:44:42.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:44:42.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:44:42.992 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:44:42.993 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:44:42.993 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:44:42.993 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:44:42.993 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:44:43.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:44:43.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:44:43.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:44:43.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:44:43.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:44:43.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:44:43.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:44:43.256 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T12:44:43.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:44:43.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:44:43.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:44:43.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280514 2026-02-19T12:44:43.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280514 2026-02-19T12:44:43.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-94489280514' 2026-02-19T12:44:43.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:44:43.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:44:43.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705678 2026-02-19T12:44:43.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705678 2026-02-19T12:44:43.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-94489280514 1-38654705678' 2026-02-19T12:44:43.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:44:43.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-94489280514 2026-02-19T12:44:43.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:44:43.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:44:43.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-94489280514 2026-02-19T12:44:43.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:44:43.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280514 2026-02-19T12:44:43.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 94489280514' 2026-02-19T12:44:43.406 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 94489280514 2026-02-19T12:44:43.406 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:44:43.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836494 -lt 94489280514 2026-02-19T12:44:43.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:44:44.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:44:44.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:44:44.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280515 -lt 94489280514 2026-02-19T12:44:44.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:44:44.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705678 2026-02-19T12:44:44.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:44:44.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:44:44.732 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705678 2026-02-19T12:44:44.732 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:44:44.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705678 2026-02-19T12:44:44.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705678' 2026-02-19T12:44:44.734 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 38654705678 2026-02-19T12:44:44.734 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:44:44.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705678 -lt 38654705678 2026-02-19T12:44:44.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:44:44.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:44:44.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:44:45.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:44:45.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:44:45.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:44:45.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:44:45.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:44:45.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:44:45.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:44:45.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:44:45.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:44:45.249 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:44:45.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:44:45.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:44:45.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:44:45.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:44:45.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:44:45.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: oper_scrub_and_schedule: rados --pool rbd get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:44:45.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: oper_scrub_and_schedule: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:44:45.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T12:44:45.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:44:45.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:44:45.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:44:45.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:44:45.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:44:45.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:44:45.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:44:45.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:44:45.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:44:45.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:44:45.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:44:45.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:44:45.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:44:45.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:44:45.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:44:45.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:44:45.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:44:45.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:44:45.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:44:45.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:44:45.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:44:45.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:44:45.794 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:44:45.794 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:45.794 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:44:45.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:44:45.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:44:45.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:44:45.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T12:44:45.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T12:44:45.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T12:44:45.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T12:44:45.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:44:45.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:44:45.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:44:45.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:44:45.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:44:45.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:44:45.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:44:45.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:44:45.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:44:45.800 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:44:45.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:44:45.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:44:45.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:44:45.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:44:45.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:44:45.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:44:45.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:44:45.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:44:45.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:44:45.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:44:45.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:44:45.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:44:45.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:44:45.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:45.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:44:45.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:44:45.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:44:45.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:44:45.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T12:44:45.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:44:45.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:45.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:44:45.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T12:44:45.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:44:45.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T12:44:45.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:44:45.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T12:44:45.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_allow_repair_during_recovery td/osd-scrub-repair 2026-02-19T12:44:45.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:138: TEST_allow_repair_during_recovery: local dir=td/osd-scrub-repair 2026-02-19T12:44:45.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:139: TEST_allow_repair_during_recovery: local poolname=rbd 2026-02-19T12:44:45.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:141: TEST_allow_repair_during_recovery: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-19T12:44:45.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T12:44:45.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T12:44:45.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T12:44:45.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T12:44:45.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T12:44:45.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-19T12:44:45.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T12:44:45.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:44:45.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:44:45.837 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:44:45.837 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:45.838 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:44:45.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:44:45.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-19T12:44:45.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T12:44:45.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T12:44:45.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:44:45.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:44:45.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T12:44:45.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T12:44:45.867 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:44:45.867 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:44:45.867 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:44:45.867 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:44:45.867 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:45.867 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:44:45.867 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:44:45.868 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:44:45.868 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T12:44:45.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T12:44:45.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:44:45.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:44:45.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T12:44:45.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T12:44:45.923 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:44:45.923 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:44:45.923 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:44:45.924 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:44:45.924 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:45.924 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:44:45.924 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:44:45.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:44:45.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T12:44:45.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:142: TEST_allow_repair_during_recovery: run_mgr td/osd-scrub-repair x 2026-02-19T12:44:45.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T12:44:45.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T12:44:45.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T12:44:45.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T12:44:45.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T12:44:45.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T12:44:46.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T12:44:46.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:44:46.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:44:46.073 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:44:46.073 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:46.073 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:44:46.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:44:46.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:44:46.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:44:46.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:143: TEST_allow_repair_during_recovery: run_osd td/osd-scrub-repair 0 --osd_scrub_during_recovery=false --osd_debug_pretend_recovery_active=true 2026-02-19T12:44:46.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:44:46.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:44:46.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T12:44:46.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:44:46.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:44:46.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:44:46.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:44:46.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:44:46.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:44:46.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:44:46.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:44:46.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:44:46.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:44:46.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:44:46.104 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:44:46.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:44:46.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:44:46.105 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:44:46.105 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:46.105 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:44:46.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:44:46.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:44:46.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:44:46.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:44:46.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:44:46.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:44:46.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:44:46.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:44:46.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:44:46.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:44:46.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:44:46.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:44:46.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_scrub_during_recovery=false --osd_debug_pretend_recovery_active=true' 2026-02-19T12:44:46.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:44:46.107 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:44:46.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6910b178-502f-41bc-96e1-0a6a4ad50250 2026-02-19T12:44:46.107 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 6910b178-502f-41bc-96e1-0a6a4ad50250 2026-02-19T12:44:46.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 6910b178-502f-41bc-96e1-0a6a4ad50250' 2026-02-19T12:44:46.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:44:46.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC+BZdpGWPpBhAA9vydfBVJEwHH6TwO/FE/kw== 2026-02-19T12:44:46.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC+BZdpGWPpBhAA9vydfBVJEwHH6TwO/FE/kw=="}' 2026-02-19T12:44:46.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6910b178-502f-41bc-96e1-0a6a4ad50250 -i td/osd-scrub-repair/0/new.json 2026-02-19T12:44:46.207 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:44:46.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T12:44:46.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_debug_pretend_recovery_active=true --mkfs --key AQC+BZdpGWPpBhAA9vydfBVJEwHH6TwO/FE/kw== --osd-uuid 6910b178-502f-41bc-96e1-0a6a4ad50250 2026-02-19T12:44:46.232 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:46.230+0000 7f9cc7935a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:46.234 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:46.230+0000 7f9cc7935a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:46.235 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:46.230+0000 7f9cc7935a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:46.236 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:46.230+0000 7f9cc7935a40 -1 bdev(0x55c10ab60c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:44:46.236 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:46.230+0000 7f9cc7935a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T12:44:47.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T12:44:47.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:44:47.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T12:44:47.065 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:44:47.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:44:47.266 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:44:47.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T12:44:47.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_debug_pretend_recovery_active=true 2026-02-19T12:44:47.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:44:47.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:44:47.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:44:47.290 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:47.286+0000 7f891a856a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:47.292 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:47.286+0000 7f891a856a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:47.293 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:47.290+0000 7f891a856a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:47.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T12:44:47.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:44:47.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:44:47.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:44:47.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:44:47.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:44:47.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:44:47.428 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:44:47.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:44:47.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:44:47.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:44:47.658 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:47.654+0000 7f891a856a40 -1 Falling back to public interface 2026-02-19T12:44:48.240 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:48.234+0000 7f891a856a40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:44:48.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:44:48.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:44:48.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:44:48.581 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:44:48.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:44:48.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:44:48.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:44:49.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:44:49.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:44:49.747 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:44:49.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:44:49.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:44:49.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:44:49.904 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2712722885,v1:127.0.0.1:6803/2712722885] [v2:127.0.0.1:6804/2712722885,v1:127.0.0.1:6805/2712722885] exists,up 6910b178-502f-41bc-96e1-0a6a4ad50250 2026-02-19T12:44:49.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:44:49.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:44:49.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:44:49.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:145: TEST_allow_repair_during_recovery: run_osd td/osd-scrub-repair 1 --osd_scrub_during_recovery=false --osd_debug_pretend_recovery_active=true 2026-02-19T12:44:49.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:44:49.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:44:49.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T12:44:49.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:44:49.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:44:49.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:44:49.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:44:49.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:44:49.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:44:49.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:44:49.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:44:49.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:44:49.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:44:49.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:44:49.908 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:44:49.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:44:49.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:44:49.909 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:44:49.909 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:49.909 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:44:49.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:44:49.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:44:49.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:44:49.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:44:49.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:44:49.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:44:49.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:44:49.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:44:49.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:44:49.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:44:49.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:44:49.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:44:49.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_scrub_during_recovery=false --osd_debug_pretend_recovery_active=true' 2026-02-19T12:44:49.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:44:49.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:44:49.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c851103d-c068-4762-80bd-debf402a92e2 2026-02-19T12:44:49.911 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 c851103d-c068-4762-80bd-debf402a92e2 2026-02-19T12:44:49.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 c851103d-c068-4762-80bd-debf402a92e2' 2026-02-19T12:44:49.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:44:49.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDBBZdpC73FNhAAg17hj5Sa7CaWy8W2lkfOnA== 2026-02-19T12:44:49.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDBBZdpC73FNhAAg17hj5Sa7CaWy8W2lkfOnA=="}' 2026-02-19T12:44:49.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c851103d-c068-4762-80bd-debf402a92e2 -i td/osd-scrub-repair/1/new.json 2026-02-19T12:44:50.081 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:44:50.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T12:44:50.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_debug_pretend_recovery_active=true --mkfs --key AQDBBZdpC73FNhAAg17hj5Sa7CaWy8W2lkfOnA== --osd-uuid c851103d-c068-4762-80bd-debf402a92e2 2026-02-19T12:44:50.107 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:50.102+0000 7fdcc1277a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:50.109 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:50.106+0000 7fdcc1277a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:50.110 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:50.106+0000 7fdcc1277a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:50.110 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:50.106+0000 7fdcc1277a40 -1 bdev(0x5623c0220c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:44:50.110 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:50.106+0000 7fdcc1277a40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T12:44:50.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T12:44:50.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:44:50.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T12:44:50.769 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:44:50.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:44:50.967 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:44:50.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T12:44:50.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_debug_pretend_recovery_active=true 2026-02-19T12:44:50.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:44:50.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:44:50.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:44:50.992 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:50.986+0000 7f1d9664ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:50.994 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:50.990+0000 7f1d9664ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:50.995 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:50.990+0000 7f1d9664ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:51.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T12:44:51.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:44:51.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:44:51.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:44:51.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:44:51.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:44:51.129 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:44:51.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:44:51.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:44:51.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:44:51.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:44:51.390 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:51.386+0000 7f1d9664ea40 -1 Falling back to public interface 2026-02-19T12:44:51.944 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:51.938+0000 7f1d9664ea40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:44:52.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:44:52.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:44:52.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:44:52.285 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:44:52.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:44:52.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:44:52.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:44:53.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:44:53.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:44:53.441 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:44:53.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:44:53.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:44:53.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:44:53.595 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3916879258,v1:127.0.0.1:6811/3916879258] [v2:127.0.0.1:6812/3916879258,v1:127.0.0.1:6813/3916879258] exists,up c851103d-c068-4762-80bd-debf402a92e2 2026-02-19T12:44:53.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:44:53.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:44:53.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:44:53.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:147: TEST_allow_repair_during_recovery: create_rbd_pool 2026-02-19T12:44:53.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T12:44:53.742 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T12:44:53.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T12:44:53.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T12:44:53.959 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T12:44:53.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:44:54.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T12:44:55.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:148: TEST_allow_repair_during_recovery: wait_for_clean 2026-02-19T12:44:55.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:44:55.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:44:55.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:44:55.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:44:55.252 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:44:55.252 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:44:55.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:44:55.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:44:55.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:44:55.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:44:55.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:44:55.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:44:55.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:44:55.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:44:55.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:44:55.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:44:55.520 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T12:44:55.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:44:55.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:44:55.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:44:55.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T12:44:55.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T12:44:55.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T12:44:55.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:44:55.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:44:55.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705666 2026-02-19T12:44:55.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705666 2026-02-19T12:44:55.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705666' 2026-02-19T12:44:55.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:44:55.665 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T12:44:55.665 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:44:55.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:44:55.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T12:44:55.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:44:55.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T12:44:55.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T12:44:55.668 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T12:44:55.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:44:55.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T12:44:55.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:44:56.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:44:56.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:44:56.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-19T12:44:56.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:44:56.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705666 2026-02-19T12:44:56.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:44:56.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:44:56.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705666 2026-02-19T12:44:56.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:44:56.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705666 2026-02-19T12:44:56.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705666' 2026-02-19T12:44:56.997 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705666 2026-02-19T12:44:56.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:44:57.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705666 -lt 38654705666 2026-02-19T12:44:57.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:44:57.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:44:57.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:44:57.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:44:57.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:44:57.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:44:57.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:44:57.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:44:57.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:44:57.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:44:57.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:44:57.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:44:57.511 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:44:57.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:44:57.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:44:57.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:44:57.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:44:57.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:44:57.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:150: TEST_allow_repair_during_recovery: add_something td/osd-scrub-repair rbd 2026-02-19T12:44:57.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T12:44:57.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=rbd 2026-02-19T12:44:57.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-19T12:44:57.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T12:44:57.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T12:44:57.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T12:44:57.909 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T12:44:57.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T12:44:58.114 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T12:44:58.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T12:44:58.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T12:44:58.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool rbd put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-19T12:44:58.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:151: TEST_allow_repair_during_recovery: get_not_primary rbd SOMETHING 2026-02-19T12:44:58.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=rbd 2026-02-19T12:44:58.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:44:58.154 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary rbd SOMETHING 2026-02-19T12:44:58.154 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-19T12:44:58.154 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:44:58.155 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd SOMETHING 2026-02-19T12:44:58.155 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:44:58.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:44:58.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map rbd SOMETHING 2026-02-19T12:44:58.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:44:58.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:151: TEST_allow_repair_during_recovery: corrupt_and_repair_one td/osd-scrub-repair rbd 0 2026-02-19T12:44:58.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:286: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-19T12:44:58.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:287: corrupt_and_repair_one: local poolname=rbd 2026-02-19T12:44:58.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:288: corrupt_and_repair_one: local osd=0 2026-02-19T12:44:58.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:293: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T12:44:58.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:44:58.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:44:58.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:44:58.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:44:58.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T12:44:58.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:44:58.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:44:58.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:44:58.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:44:58.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:44:58.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:44:58.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:44:58.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:44:58.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:44:58.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:44:58.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:44:58.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T12:44:58.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:44:58.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:44:58.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:44:58.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:44:58.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:44:58.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-19T12:44:58.973 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-19T12:44:59.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T12:44:59.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:44:59.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:44:59.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:44:59.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:44:59.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:44:59.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:44:59.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:44:59.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:44:59.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:44:59.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:44:59.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:44:59.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:44:59.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:44:59.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:44:59.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:44:59.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:44:59.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:44:59.061 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:44:59.061 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:44:59.061 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:44:59.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:44:59.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:44:59.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:44:59.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:44:59.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:44:59.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:44:59.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:44:59.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:44:59.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:44:59.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:44:59.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:44:59.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:44:59.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:44:59.064 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:44:59.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:44:59.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:44:59.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:44:59.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:44:59.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:44:59.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:44:59.080 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:59.074+0000 7f8c33c57a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:59.082 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:59.078+0000 7f8c33c57a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:59.083 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:59.078+0000 7f8c33c57a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:44:59.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:44:59.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:44:59.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:44:59.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:44:59.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:44:59.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:44:59.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:44:59.222 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:44:59.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:44:59.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:44:59.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:44:59.462 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:44:59.458+0000 7f8c33c57a40 -1 Falling back to public interface 2026-02-19T12:45:00.228 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:00.222+0000 7f8c33c57a40 -1 osd.0 17 log_to_monitors true 2026-02-19T12:45:00.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:45:00.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:45:00.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:45:00.384 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:45:00.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:45:00.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:45:00.541 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 21 up_thru 21 down_at 18 last_clean_interval [5,17) [v2:127.0.0.1:6802/403302163,v1:127.0.0.1:6803/403302163] [v2:127.0.0.1:6804/403302163,v1:127.0.0.1:6805/403302163] exists,up 6910b178-502f-41bc-96e1-0a6a4ad50250 2026-02-19T12:45:00.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:45:00.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:45:00.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:45:00.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:45:00.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:45:00.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:45:00.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:45:00.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:45:00.543 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:45:00.543 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:45:00.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:45:00.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:45:00.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:45:00.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:45:00.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:45:00.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:45:00.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:45:00.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:45:00.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:45:00.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:45:00.815 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T12:45:00.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:45:00.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:00.816 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:45:00.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313218 2026-02-19T12:45:00.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313218 2026-02-19T12:45:00.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-90194313218' 2026-02-19T12:45:00.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:00.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:45:00.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705668 2026-02-19T12:45:00.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705668 2026-02-19T12:45:00.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-90194313218 1-38654705668' 2026-02-19T12:45:00.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:00.966 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-90194313218 2026-02-19T12:45:00.967 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:00.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:45:00.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-90194313218 2026-02-19T12:45:00.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:00.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313218 2026-02-19T12:45:00.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 90194313218' 2026-02-19T12:45:00.970 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 90194313218 2026-02-19T12:45:00.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:01.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 90194313218 2026-02-19T12:45:01.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:45:02.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:45:02.133 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:02.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 90194313218 2026-02-19T12:45:02.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:45:03.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:45:03.294 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:03.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313218 -lt 90194313218 2026-02-19T12:45:03.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:03.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705668 2026-02-19T12:45:03.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:03.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:45:03.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705668 2026-02-19T12:45:03.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:03.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705668 2026-02-19T12:45:03.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-19T12:45:03.459 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 38654705668 2026-02-19T12:45:03.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:45:03.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-19T12:45:03.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:45:03.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:45:03.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:45:03.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:45:03.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:45:03.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:45:03.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:45:03.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:45:03.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:45:03.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:45:03.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:45:03.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:45:03.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:45:03.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:45:03.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:45:04.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:45:04.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:45:04.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:45:04.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: get_pg rbd SOMETHING 2026-02-19T12:45:04.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=rbd 2026-02-19T12:45:04.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:45:04.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map rbd SOMETHING 2026-02-19T12:45:04.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:45:04.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: local pg=1.3 2026-02-19T12:45:04.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: corrupt_and_repair_one: repair 1.3 2026-02-19T12:45:04.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=1.3 2026-02-19T12:45:04.329 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 1.3 2026-02-19T12:45:04.329 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:45:04.329 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:45:04.329 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:45:04.329 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:45:04.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:44:53.957754+0000 2026-02-19T12:45:04.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 1.3 2026-02-19T12:45:04.632 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.3 on osd.1 to repair 2026-02-19T12:45:04.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 1.3 2026-02-19T12:44:53.957754+0000 2026-02-19T12:45:04.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.3 2026-02-19T12:45:04.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:44:53.957754+0000 2026-02-19T12:45:04.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:45:04.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:45:04.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:45:04.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T12:45:04.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:45:04.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:45:04.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:45:04.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:45:04.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:44:53.957754+0000 '>' 2026-02-19T12:44:53.957754+0000 2026-02-19T12:45:04.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:45:05.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:45:05.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:45:05.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T12:45:05.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:45:05.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:45:05.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:45:05.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:45:05.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:44:53.957754+0000 '>' 2026-02-19T12:44:53.957754+0000 2026-02-19T12:45:05.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:45:06.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:45:06.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:45:06.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T12:45:06.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:45:06.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:45:06.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:45:06.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:45:07.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:44:53.957754+0000 '>' 2026-02-19T12:44:53.957754+0000 2026-02-19T12:45:07.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:45:08.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:45:08.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:45:08.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T12:45:08.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:45:08.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:45:08.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:45:08.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:45:08.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:44:53.957754+0000 '>' 2026-02-19T12:44:53.957754+0000 2026-02-19T12:45:08.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:45:09.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:45:09.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:45:09.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T12:45:09.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:45:09.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:45:09.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:45:09.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:45:09.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:45:05.101813+0000 '>' 2026-02-19T12:44:53.957754+0000 2026-02-19T12:45:09.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:45:09.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:302: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:45:09.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:45:09.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:45:09.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:45:09.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:45:09.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:45:09.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:45:09.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:45:09.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:45:09.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:45:09.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:45:09.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:45:09.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:45:09.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:45:09.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:45:09.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:45:09.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:45:09.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:45:09.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:45:09.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:45:09.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:45:09.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:45:09.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:45:09.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-19T12:45:09.815 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:45:09.816 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:45:09.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:45:09.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:45:09.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:45:09.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:45:09.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:45:09.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:45:09.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:45:09.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:45:09.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:45:09.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:45:09.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:45:09.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:45:09.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:45:09.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:45:09.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:45:09.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:45:09.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:45:09.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:45:09.897 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:45:09.897 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:45:09.897 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:45:09.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:45:09.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:45:09.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:45:09.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:45:09.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:45:09.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:45:09.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:45:09.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:45:09.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:45:09.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:45:09.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:45:09.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:45:09.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:45:09.899 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:45:09.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:45:09.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:45:09.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:45:09.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:45:09.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:45:09.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:45:09.916 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:09.909+0000 7f23d09fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:09.917 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:09.913+0000 7f23d09fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:09.918 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:09.913+0000 7f23d09fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:10.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:45:10.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:45:10.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:45:10.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:45:10.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:45:10.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:45:10.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:45:10.054 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:45:10.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:45:10.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:45:10.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:45:10.266 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:10.261+0000 7f23d09fca40 -1 Falling back to public interface 2026-02-19T12:45:10.886 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:10.881+0000 7f23d09fca40 -1 osd.1 25 log_to_monitors true 2026-02-19T12:45:11.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:45:11.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:45:11.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:45:11.217 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:45:11.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:45:11.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:45:11.376 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 28 up_thru 28 down_at 25 last_clean_interval [9,25) [v2:127.0.0.1:6810/2716461414,v1:127.0.0.1:6811/2716461414] [v2:127.0.0.1:6812/2716461414,v1:127.0.0.1:6813/2716461414] exists,up c851103d-c068-4762-80bd-debf402a92e2 2026-02-19T12:45:11.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:45:11.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:45:11.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:45:11.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:45:11.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:45:11.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:45:11.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:45:11.376 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:45:11.377 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:45:11.377 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:45:11.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:45:11.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:45:11.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:45:11.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:45:11.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:45:11.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:45:11.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:45:11.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:45:11.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:45:11.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:45:11.647 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T12:45:11.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:45:11.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:11.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:45:11.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313221 2026-02-19T12:45:11.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313221 2026-02-19T12:45:11.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-90194313221' 2026-02-19T12:45:11.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:11.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:45:11.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084290 2026-02-19T12:45:11.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084290 2026-02-19T12:45:11.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-90194313221 1-120259084290' 2026-02-19T12:45:11.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:11.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-90194313221 2026-02-19T12:45:11.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:11.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:45:11.793 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-90194313221 2026-02-19T12:45:11.793 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:11.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313221 2026-02-19T12:45:11.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 90194313221' 2026-02-19T12:45:11.795 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 90194313221 2026-02-19T12:45:11.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:11.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313220 -lt 90194313221 2026-02-19T12:45:11.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:45:12.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:45:12.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:13.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313221 -lt 90194313221 2026-02-19T12:45:13.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:13.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-120259084290 2026-02-19T12:45:13.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:13.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:45:13.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-120259084290 2026-02-19T12:45:13.131 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:13.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=120259084290 2026-02-19T12:45:13.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 120259084290' 2026-02-19T12:45:13.132 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 120259084290 2026-02-19T12:45:13.133 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:45:13.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084290 -lt 120259084290 2026-02-19T12:45:13.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:45:13.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:45:13.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:45:13.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:45:13.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:45:13.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:45:13.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:45:13.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:45:13.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:45:13.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:45:13.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:45:13.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:45:13.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:45:13.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:45:13.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:45:13.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:45:13.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:45:13.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:45:13.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: corrupt_and_repair_one: rados --pool rbd get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:45:13.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:304: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:45:13.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T12:45:13.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:45:13.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:45:13.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:45:13.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:45:13.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:45:13.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:45:13.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:45:13.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:45:13.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:45:13.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:45:13.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:45:13.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:45:13.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:45:13.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:45:13.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:45:13.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:45:13.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:45:13.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:45:13.978 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:45:13.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:45:13.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:45:13.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:45:13.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:45:13.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:45:13.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:45:13.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:45:13.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:45:13.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:45:13.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T12:45:13.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T12:45:13.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T12:45:13.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T12:45:13.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:45:13.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:45:13.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:45:13.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:45:13.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:45:13.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:45:13.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:45:13.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:45:13.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:45:13.995 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:45:13.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:45:13.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:45:13.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:45:13.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:45:13.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:45:13.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:45:13.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:45:13.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:45:13.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:45:14.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:45:14.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:45:14.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:45:14.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:45:14.003 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:45:14.003 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:45:14.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:45:14.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:45:14.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:45:14.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T12:45:14.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:45:14.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:45:14.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:45:14.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T12:45:14.008 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:45:14.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T12:45:14.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:45:14.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T12:45:14.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_bluestore_basic td/osd-scrub-repair 2026-02-19T12:45:14.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:498: TEST_auto_repair_bluestore_basic: local dir=td/osd-scrub-repair 2026-02-19T12:45:14.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:499: TEST_auto_repair_bluestore_basic: cluster_conf=(['osds_num']='3' ['pgs_in_pool']='1' ['pool_name']='testpool' ['extras']=' --osd_scrub_auto_repair=true') 2026-02-19T12:45:14.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:499: TEST_auto_repair_bluestore_basic: local -A cluster_conf 2026-02-19T12:45:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:505: TEST_auto_repair_bluestore_basic: standard_scrub_cluster td/osd-scrub-repair cluster_conf 2026-02-19T12:45:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:229: standard_scrub_cluster: local dir=td/osd-scrub-repair 2026-02-19T12:45:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:230: standard_scrub_cluster: local -n args=cluster_conf 2026-02-19T12:45:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:232: standard_scrub_cluster: local OSDS=3 2026-02-19T12:45:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:233: standard_scrub_cluster: local pg_num=1 2026-02-19T12:45:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:235: standard_scrub_cluster: local poolname=testpool 2026-02-19T12:45:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:236: standard_scrub_cluster: args['pool_name']=testpool 2026-02-19T12:45:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:238: standard_scrub_cluster: local pool_default_size=3 2026-02-19T12:45:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:239: standard_scrub_cluster: args['pool_default_size']=3 2026-02-19T12:45:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:241: standard_scrub_cluster: local 'extra_pars= --osd_scrub_auto_repair=true' 2026-02-19T12:45:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:242: standard_scrub_cluster: local debug_msg=dbg 2026-02-19T12:45:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:245: standard_scrub_cluster: local saved_echo_flag=x 2026-02-19T12:45:14.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:246: standard_scrub_cluster: set +x 2026-02-19T12:45:14.299 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 cc0092b2-98dc-4fc1-9fe3-376d5d3d26d2 2026-02-19T12:45:14.399 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:45:14.429 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:14.425+0000 7f9ecc8a2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:14.430 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:14.425+0000 7f9ecc8a2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:14.431 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:14.425+0000 7f9ecc8a2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:14.431 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:14.425+0000 7f9ecc8a2a40 -1 bdev(0x55a70beacc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:45:14.432 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:14.425+0000 7f9ecc8a2a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T12:45:15.180 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:45:15.383 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:45:15.417 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:15.414+0000 7f4cb3481a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:15.418 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:15.414+0000 7f4cb3481a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:15.419 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:15.414+0000 7f4cb3481a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:15.538 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:45:15.834 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:15.830+0000 7f4cb3481a40 -1 Falling back to public interface 2026-02-19T12:45:16.427 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:16.422+0000 7f4cb3481a40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:45:16.699 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:45:17.861 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:45:18.018 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1417737625,v1:127.0.0.1:6803/1417737625] [v2:127.0.0.1:6804/1417737625,v1:127.0.0.1:6805/1417737625] exists,up cc0092b2-98dc-4fc1-9fe3-376d5d3d26d2 2026-02-19T12:45:18.023 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 7448caa5-f1fa-488f-b4eb-8ab048c2b7bd 2026-02-19T12:45:18.195 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:45:18.225 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:18.222+0000 7f923f128a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:18.227 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:18.222+0000 7f923f128a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:18.228 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:18.222+0000 7f923f128a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:18.228 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:18.226+0000 7f923f128a40 -1 bdev(0x55cbc034cc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:45:18.229 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:18.226+0000 7f923f128a40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T12:45:18.897 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:45:19.093 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:45:19.124 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:19.122+0000 7fd615d22a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:19.126 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:19.122+0000 7fd615d22a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:19.128 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:19.122+0000 7fd615d22a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:19.253 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:45:19.570 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:19.566+0000 7fd615d22a40 -1 Falling back to public interface 2026-02-19T12:45:20.122 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:20.117+0000 7fd615d22a40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:45:20.411 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:45:21.571 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:45:21.726 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1567116524,v1:127.0.0.1:6811/1567116524] [v2:127.0.0.1:6812/1567116524,v1:127.0.0.1:6813/1567116524] exists,up 7448caa5-f1fa-488f-b4eb-8ab048c2b7bd 2026-02-19T12:45:21.731 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 86eb9fc2-144f-487f-a066-df35c4eb1bbf 2026-02-19T12:45:21.906 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:45:21.938 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:21.933+0000 7fd085389a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:21.940 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:21.933+0000 7fd085389a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:21.940 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:21.937+0000 7fd085389a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:21.941 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:21.937+0000 7fd085389a40 -1 bdev(0x564b95920c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T12:45:21.941 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:21.937+0000 7fd085389a40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T12:45:22.556 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T12:45:22.754 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T12:45:22.787 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:22.781+0000 7f4c248b6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:22.788 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:22.785+0000 7f4c248b6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:22.789 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:22.785+0000 7f4c248b6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:22.911 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:45:23.202 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:23.197+0000 7f4c248b6a40 -1 Falling back to public interface 2026-02-19T12:45:23.863 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:23.857+0000 7f4c248b6a40 -1 osd.2 0 log_to_monitors true 2026-02-19T12:45:24.074 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:45:25.253 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:45:25.409 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/898706312,v1:127.0.0.1:6819/898706312] [v2:127.0.0.1:6820/898706312,v1:127.0.0.1:6821/898706312] exists,up 86eb9fc2-144f-487f-a066-df35c4eb1bbf 2026-02-19T12:45:25.614 INFO:tasks.workunit.client.0.trial144.stderr:pool 'testpool' created 2026-02-19T12:45:27.111 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836491 2026-02-19T12:45:29.601 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705671 2026-02-19T12:45:29.761 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574852 2026-02-19T12:45:31.651 INFO:tasks.workunit.client.0.trial144.stdout:standard_scrub_cluster: dbg: test pool is testpool 1 2026-02-19T12:45:31.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:506: TEST_auto_repair_bluestore_basic: local poolid=1 2026-02-19T12:45:31.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:507: TEST_auto_repair_bluestore_basic: local poolname=testpool 2026-02-19T12:45:31.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:509: TEST_auto_repair_bluestore_basic: ceph osd pool set testpool size 2 2026-02-19T12:45:31.852 INFO:tasks.workunit.client.0.trial144.stderr:set pool 1 size to 2 2026-02-19T12:45:31.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:510: TEST_auto_repair_bluestore_basic: wait_for_clean 2026-02-19T12:45:31.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:45:31.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:45:31.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:45:31.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:45:31.866 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:45:31.866 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:45:31.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:45:31.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:45:31.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:45:31.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:45:31.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:45:31.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:45:31.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:45:31.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:45:31.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:45:32.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:45:32.134 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:45:32.134 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:45:32.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:45:32.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:32.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:45:32.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836497 2026-02-19T12:45:32.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836497 2026-02-19T12:45:32.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497' 2026-02-19T12:45:32.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:32.205 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:45:32.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705677 2026-02-19T12:45:32.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705677 2026-02-19T12:45:32.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-38654705677' 2026-02-19T12:45:32.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:32.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:45:32.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574858 2026-02-19T12:45:32.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574858 2026-02-19T12:45:32.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-38654705677 2-55834574858' 2026-02-19T12:45:32.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:32.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836497 2026-02-19T12:45:32.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:32.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:45:32.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836497 2026-02-19T12:45:32.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:32.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836497 2026-02-19T12:45:32.356 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836497 2026-02-19T12:45:32.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2026-02-19T12:45:32.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:32.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836495 -lt 21474836497 2026-02-19T12:45:32.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:45:33.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:45:33.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:33.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836498 -lt 21474836497 2026-02-19T12:45:33.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:33.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705677 2026-02-19T12:45:33.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:33.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:45:33.682 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705677 2026-02-19T12:45:33.682 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:33.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705677 2026-02-19T12:45:33.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705677' 2026-02-19T12:45:33.684 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705677 2026-02-19T12:45:33.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:45:33.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705678 -lt 38654705677 2026-02-19T12:45:33.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:33.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574858 2026-02-19T12:45:33.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:33.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:45:33.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574858 2026-02-19T12:45:33.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:33.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574858 2026-02-19T12:45:33.863 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574858 2026-02-19T12:45:33.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574858' 2026-02-19T12:45:33.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:45:34.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574858 -lt 55834574858 2026-02-19T12:45:34.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:45:34.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:45:34.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:45:34.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:45:34.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:45:34.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:45:34.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:45:34.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:45:34.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:45:34.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:45:34.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:45:34.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:45:34.376 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:45:34.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:45:34.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:45:34.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:45:34.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:45:34.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:45:34.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:513: TEST_auto_repair_bluestore_basic: local payload=ABCDEF 2026-02-19T12:45:34.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:514: TEST_auto_repair_bluestore_basic: echo ABCDEF 2026-02-19T12:45:34.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:515: TEST_auto_repair_bluestore_basic: rados --pool testpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-19T12:45:34.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:519: TEST_auto_repair_bluestore_basic: get_not_primary testpool SOMETHING 2026-02-19T12:45:34.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=testpool 2026-02-19T12:45:34.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:45:34.592 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary testpool SOMETHING 2026-02-19T12:45:34.592 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:45:34.592 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:45:34.592 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:45:34.592 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:45:34.760 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:45:34.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:45:34.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:45:34.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:519: TEST_auto_repair_bluestore_basic: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T12:45:34.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:45:34.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:45:34.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:45:34.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:45:34.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T12:45:34.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:45:34.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:45:34.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:45:34.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:45:34.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:45:34.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:45:34.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:45:34.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:45:34.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:45:34.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:45:35.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:45:35.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T12:45:35.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:45:35.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:45:35.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:45:35.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:45:35.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:45:35.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-19T12:45:35.399 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-19T12:45:35.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T12:45:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:45:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:45:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:45:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:45:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:45:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:45:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:45:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:45:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:45:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:45:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:45:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:45:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:45:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:45:35.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:45:35.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:45:35.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:45:35.501 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:45:35.501 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:45:35.501 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:45:35.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:45:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:45:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:45:35.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:45:35.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:45:35.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:45:35.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:45:35.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:45:35.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:45:35.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:45:35.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:45:35.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:45:35.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:45:35.502 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:45:35.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:45:35.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:45:35.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:45:35.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:45:35.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:45:35.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:45:35.519 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:35.513+0000 7f913697ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:35.521 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:35.517+0000 7f913697ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:35.521 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:35.517+0000 7f913697ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:35.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:45:35.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:45:35.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:45:35.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:45:35.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:45:35.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:45:35.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:45:35.657 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:45:35.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:45:35.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:45:35.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:45:35.910 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:35.905+0000 7f913697ca40 -1 Falling back to public interface 2026-02-19T12:45:36.554 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:36.549+0000 7f913697ca40 -1 osd.0 21 log_to_monitors true 2026-02-19T12:45:36.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:45:36.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:45:36.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:45:36.822 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:45:36.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:45:36.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:45:36.979 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 25 up_thru 0 down_at 22 last_clean_interval [5,21) [v2:127.0.0.1:6802/3328515408,v1:127.0.0.1:6803/3328515408] [v2:127.0.0.1:6804/3328515408,v1:127.0.0.1:6805/3328515408] exists,up cc0092b2-98dc-4fc1-9fe3-376d5d3d26d2 2026-02-19T12:45:36.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:45:36.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:45:36.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:45:36.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:45:36.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:45:36.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:45:36.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:45:36.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:45:36.982 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:45:36.982 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:45:36.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:45:36.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:45:36.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:45:37.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:45:37.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:45:37.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:45:37.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:45:37.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:45:37.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:45:37.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:45:37.247 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:45:37.247 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:45:37.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:45:37.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:37.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:45:37.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182402 2026-02-19T12:45:37.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182402 2026-02-19T12:45:37.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-107374182402' 2026-02-19T12:45:37.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:37.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:45:37.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705684 2026-02-19T12:45:37.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705684 2026-02-19T12:45:37.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-107374182402 1-38654705684' 2026-02-19T12:45:37.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:37.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:45:37.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574864 2026-02-19T12:45:37.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574864 2026-02-19T12:45:37.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-107374182402 1-38654705684 2-55834574864' 2026-02-19T12:45:37.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:37.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-107374182402 2026-02-19T12:45:37.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:37.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:45:37.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-107374182402 2026-02-19T12:45:37.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:37.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182402 2026-02-19T12:45:37.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 107374182402' 2026-02-19T12:45:37.471 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 107374182402 2026-02-19T12:45:37.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:37.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 107374182402 2026-02-19T12:45:37.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:45:38.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:45:38.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:38.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 107374182402 2026-02-19T12:45:38.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:45:39.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:45:39.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:39.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182404 -lt 107374182402 2026-02-19T12:45:39.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:39.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705684 2026-02-19T12:45:39.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:39.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:45:39.961 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705684 2026-02-19T12:45:39.961 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:39.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705684 2026-02-19T12:45:39.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705684' 2026-02-19T12:45:39.962 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 38654705684 2026-02-19T12:45:39.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:45:40.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705685 -lt 38654705684 2026-02-19T12:45:40.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:40.131 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574864 2026-02-19T12:45:40.131 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:40.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:45:40.133 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574864 2026-02-19T12:45:40.133 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:40.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574864 2026-02-19T12:45:40.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574864' 2026-02-19T12:45:40.134 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 55834574864 2026-02-19T12:45:40.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:45:40.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574865 -lt 55834574864 2026-02-19T12:45:40.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:45:40.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:45:40.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:45:40.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:45:40.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:45:40.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:45:40.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:45:40.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:45:40.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:45:40.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:45:40.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:45:40.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:45:40.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:45:40.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:45:40.651 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:45:40.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:45:40.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:45:40.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:45:40.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:520: TEST_auto_repair_bluestore_basic: ceph tell 'osd.*' config set osd_scrub_auto_repair true 2026-02-19T12:45:40.910 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T12:45:40.910 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_auto_repair = '' (not observed, change may require restart) osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-19T12:45:40.910 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:45:40.917 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T12:45:40.917 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_auto_repair = '' (not observed, change may require restart) " 2026-02-19T12:45:40.918 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:45:40.925 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T12:45:40.925 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_auto_repair = '' (not observed, change may require restart) " 2026-02-19T12:45:40.925 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:45:40.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:522: TEST_auto_repair_bluestore_basic: get_pg testpool SOMETHING 2026-02-19T12:45:40.934 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=testpool 2026-02-19T12:45:40.934 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:45:40.934 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map testpool SOMETHING 2026-02-19T12:45:40.934 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:45:41.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:522: TEST_auto_repair_bluestore_basic: local pgid=1.0 2026-02-19T12:45:41.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:523: TEST_auto_repair_bluestore_basic: get_primary testpool SOMETHING 2026-02-19T12:45:41.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:45:41.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:45:41.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:45:41.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:45:41.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:523: TEST_auto_repair_bluestore_basic: local primary=1 2026-02-19T12:45:41.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:524: TEST_auto_repair_bluestore_basic: get_last_scrub_stamp 1.0 2026-02-19T12:45:41.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:45:41.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:45:41.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:45:41.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:45:41.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:524: TEST_auto_repair_bluestore_basic: local last_scrub_stamp=2026-02-19T12:45:25.613103+0000 2026-02-19T12:45:41.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:527: TEST_auto_repair_bluestore_basic: ceph tell 1.0 schedule-deep-scrub 2026-02-19T12:45:41.482 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T12:45:41.482 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T12:45:41.482 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T12:45:41.483 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-08T07:57:31.481921+0000" 2026-02-19T12:45:41.483 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:45:41.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:530: TEST_auto_repair_bluestore_basic: wait_for_scrub 1.0 2026-02-19T12:45:25.613103+0000 2026-02-19T12:45:41.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T12:45:41.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:45:25.613103+0000 2026-02-19T12:45:41.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:45:41.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:45:41.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:45:41.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:45:41.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:45:41.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:45:41.494 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:45:41.494 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:45:41.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:45:25.613103+0000 '>' 2026-02-19T12:45:25.613103+0000 2026-02-19T12:45:41.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:45:42.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:45:42.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:45:42.651 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:45:42.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:45:42.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:45:42.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:45:42.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:45:42.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:45:42.020687+0000 '>' 2026-02-19T12:45:25.613103+0000 2026-02-19T12:45:42.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:45:42.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:531: TEST_auto_repair_bluestore_basic: wait_for_clean 2026-02-19T12:45:42.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:45:42.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:45:42.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:45:42.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:45:42.810 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:45:42.810 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:45:42.811 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:45:42.811 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:45:42.811 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:45:42.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:45:42.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:45:42.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:45:42.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:45:42.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:45:42.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:45:43.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:45:43.074 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:45:43.074 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:45:43.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:45:43.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:43.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:45:43.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182409 2026-02-19T12:45:43.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182409 2026-02-19T12:45:43.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-107374182409' 2026-02-19T12:45:43.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:43.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:45:43.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705690 2026-02-19T12:45:43.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705690 2026-02-19T12:45:43.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-107374182409 1-38654705690' 2026-02-19T12:45:43.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:43.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:45:43.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574871 2026-02-19T12:45:43.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574871 2026-02-19T12:45:43.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-107374182409 1-38654705690 2-55834574871' 2026-02-19T12:45:43.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:43.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-107374182409 2026-02-19T12:45:43.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:43.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:45:43.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-107374182409 2026-02-19T12:45:43.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:43.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182409 2026-02-19T12:45:43.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 107374182409' 2026-02-19T12:45:43.300 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 107374182409 2026-02-19T12:45:43.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:43.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182408 -lt 107374182409 2026-02-19T12:45:43.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:45:44.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:45:44.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:44.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182408 -lt 107374182409 2026-02-19T12:45:44.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:45:45.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:45:45.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:45.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182411 -lt 107374182409 2026-02-19T12:45:45.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:45.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705690 2026-02-19T12:45:45.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:45.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:45:45.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705690 2026-02-19T12:45:45.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:45.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705690 2026-02-19T12:45:45.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705690' 2026-02-19T12:45:45.785 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705690 2026-02-19T12:45:45.786 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:45:45.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705692 -lt 38654705690 2026-02-19T12:45:45.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:45.943 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574871 2026-02-19T12:45:45.943 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:45.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:45:45.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574871 2026-02-19T12:45:45.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:45.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574871 2026-02-19T12:45:45.947 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574871 2026-02-19T12:45:45.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574871' 2026-02-19T12:45:45.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:45:46.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574872 -lt 55834574871 2026-02-19T12:45:46.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:45:46.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:45:46.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:45:46.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:45:46.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:45:46.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:45:46.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:45:46.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:45:46.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:45:46.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:45:46.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:45:46.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:45:46.466 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:45:46.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:45:46.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:45:46.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:45:46.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:45:46.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:45:46.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:532: TEST_auto_repair_bluestore_basic: ceph pg dump pgs 2026-02-19T12:45:46.802 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:45:46.802 INFO:tasks.workunit.client.0.trial144.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean 2026-02-19T12:45:42.022561+0000 21'1 26:73 [1,0] 1 [1,0] 1 21'1 2026-02-19T12:45:42.020687+0000 21'1 2026-02-19T12:45:42.020687+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:45:42.020687+0000 1 0 2026-02-19T12:45:46.802 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:45:46.802 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:45:46.803 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:45:46.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:535: TEST_auto_repair_bluestore_basic: get_not_primary testpool SOMETHING 2026-02-19T12:45:46.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=testpool 2026-02-19T12:45:46.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:45:46.812 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary testpool SOMETHING 2026-02-19T12:45:46.812 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:45:46.812 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:45:46.813 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:45:46.813 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:45:46.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:45:46.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:45:46.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:45:47.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:535: TEST_auto_repair_bluestore_basic: objectstore_tool td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-19T12:45:47.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:45:47.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:45:47.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:45:47.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:45:47.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-19T12:45:47.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:45:47.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:45:47.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:45:47.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:45:47.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:45:47.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:45:47.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:45:47.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:45:47.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:45:47.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:45:47.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:45:47.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-19T12:45:47.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:45:47.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:45:47.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:45:47.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:45:47.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:45:47.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING list-attrs 2026-02-19T12:45:47.511 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:45:47.511 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:45:47.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T12:45:47.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:45:47.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:45:47.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:45:47.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:45:47.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:45:47.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:45:47.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:45:47.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:45:47.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:45:47.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:45:47.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:45:47.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:45:47.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:45:47.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:45:47.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:45:47.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:45:47.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:45:47.604 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:45:47.605 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:45:47.605 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:45:47.605 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:45:47.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:45:47.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:45:47.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:45:47.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:45:47.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:45:47.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:45:47.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:45:47.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:45:47.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:45:47.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:45:47.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:45:47.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:45:47.607 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:45:47.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:45:47.607 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:45:47.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:45:47.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:45:47.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:45:47.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:45:47.624 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:47.617+0000 7f19afc42a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:47.626 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:47.621+0000 7f19afc42a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:47.626 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:47.621+0000 7f19afc42a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:47.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:45:47.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:45:47.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:45:47.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:45:47.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:45:47.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:45:47.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:45:47.756 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:45:47.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:45:47.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:45:47.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:45:48.029 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:48.025+0000 7f19afc42a40 -1 Falling back to public interface 2026-02-19T12:45:48.616 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:48.609+0000 7f19afc42a40 -1 osd.0 26 log_to_monitors true 2026-02-19T12:45:48.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:45:48.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:45:48.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:45:48.917 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:45:48.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:45:48.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:45:49.075 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 30 up_thru 0 down_at 27 last_clean_interval [25,26) [v2:127.0.0.1:6802/4256317177,v1:127.0.0.1:6803/4256317177] [v2:127.0.0.1:6804/4256317177,v1:127.0.0.1:6805/4256317177] exists,up cc0092b2-98dc-4fc1-9fe3-376d5d3d26d2 2026-02-19T12:45:49.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:45:49.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:45:49.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:45:49.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:45:49.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:45:49.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:45:49.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:45:49.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:45:49.077 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:45:49.077 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:45:49.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:45:49.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:45:49.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:45:49.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:45:49.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:45:49.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:45:49.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:45:49.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:45:49.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:45:49.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:45:49.349 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:45:49.349 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:45:49.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:45:49.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:49.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:45:49.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018882 2026-02-19T12:45:49.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018882 2026-02-19T12:45:49.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-128849018882' 2026-02-19T12:45:49.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:49.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:45:49.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705698 2026-02-19T12:45:49.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705698 2026-02-19T12:45:49.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-128849018882 1-38654705698' 2026-02-19T12:45:49.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:49.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:45:49.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574878 2026-02-19T12:45:49.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574878 2026-02-19T12:45:49.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-128849018882 1-38654705698 2-55834574878' 2026-02-19T12:45:49.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:49.568 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-128849018882 2026-02-19T12:45:49.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:49.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:45:49.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-128849018882 2026-02-19T12:45:49.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:49.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018882 2026-02-19T12:45:49.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 128849018882' 2026-02-19T12:45:49.572 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 128849018882 2026-02-19T12:45:49.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:49.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 128849018882 2026-02-19T12:45:49.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:45:50.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:45:50.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:50.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018884 -lt 128849018882 2026-02-19T12:45:50.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:50.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705698 2026-02-19T12:45:50.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:50.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:45:50.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705698 2026-02-19T12:45:50.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:50.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705698 2026-02-19T12:45:50.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705698' 2026-02-19T12:45:50.903 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 38654705698 2026-02-19T12:45:50.903 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:45:51.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705699 -lt 38654705698 2026-02-19T12:45:51.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:51.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574878 2026-02-19T12:45:51.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:51.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:45:51.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574878 2026-02-19T12:45:51.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:51.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574878 2026-02-19T12:45:51.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574878' 2026-02-19T12:45:51.069 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 55834574878 2026-02-19T12:45:51.070 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:45:51.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574879 -lt 55834574878 2026-02-19T12:45:51.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:45:51.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:45:51.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:45:51.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:45:51.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:45:51.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:45:51.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:45:51.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:45:51.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:45:51.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:45:51.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:45:51.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:45:51.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:45:51.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:45:51.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:45:51.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:45:51.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:45:51.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:45:51.786 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:536: TEST_auto_repair_bluestore_basic: get_not_primary testpool SOMETHING 2026-02-19T12:45:51.786 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=testpool 2026-02-19T12:45:51.786 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:45:51.786 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary testpool SOMETHING 2026-02-19T12:45:51.786 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:45:51.786 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:45:51.787 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:45:51.787 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:45:51.942 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:45:51.942 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:45:51.942 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:45:52.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:536: TEST_auto_repair_bluestore_basic: objectstore_tool td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-19T12:45:52.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:45:52.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:45:52.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:45:52.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:45:52.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-19T12:45:52.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:45:52.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:45:52.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:45:52.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:45:52.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:45:52.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:45:52.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:45:52.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:45:52.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:45:52.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:45:52.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:45:52.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-19T12:45:52.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:45:52.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:45:52.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:45:52.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:45:52.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:45:52.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:45:52.568 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:45:52.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:45:52.569 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:45:52.569 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:45:52.569 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:45:52.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:45:52.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:45:52.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:45:52.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:45:52.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:45:52.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:45:52.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:45:52.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:45:52.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:45:52.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:45:52.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:45:52.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:45:52.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:45:52.570 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:45:52.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:45:52.571 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:45:52.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:45:52.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:45:52.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:45:52.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:45:52.589 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:52.585+0000 7fea1ee2aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:52.590 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:52.585+0000 7fea1ee2aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:52.591 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:52.585+0000 7fea1ee2aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:52.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:45:52.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:45:52.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:45:52.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:45:52.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:45:52.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:45:52.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:45:52.737 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:45:52.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:45:52.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:45:52.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:45:52.998 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:52.993+0000 7fea1ee2aa40 -1 Falling back to public interface 2026-02-19T12:45:53.616 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:53.609+0000 7fea1ee2aa40 -1 osd.0 31 log_to_monitors true 2026-02-19T12:45:53.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:45:53.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:45:53.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:45:53.894 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:45:53.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:45:53.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:45:54.048 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 35 up_thru 0 down_at 32 last_clean_interval [30,31) [v2:127.0.0.1:6802/3228001792,v1:127.0.0.1:6803/3228001792] [v2:127.0.0.1:6804/3228001792,v1:127.0.0.1:6805/3228001792] exists,up cc0092b2-98dc-4fc1-9fe3-376d5d3d26d2 2026-02-19T12:45:54.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:45:54.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:45:54.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:45:54.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:45:54.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:45:54.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:45:54.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:45:54.049 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:45:54.050 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:45:54.050 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:45:54.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:45:54.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:45:54.051 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:45:54.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:45:54.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:45:54.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:45:54.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:45:54.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:45:54.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:45:54.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:45:54.311 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:45:54.311 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:45:54.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:45:54.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:54.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:45:54.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855362 2026-02-19T12:45:54.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855362 2026-02-19T12:45:54.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855362' 2026-02-19T12:45:54.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:54.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:45:54.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705704 2026-02-19T12:45:54.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705704 2026-02-19T12:45:54.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855362 1-38654705704' 2026-02-19T12:45:54.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:45:54.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:45:54.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574884 2026-02-19T12:45:54.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574884 2026-02-19T12:45:54.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855362 1-38654705704 2-55834574884' 2026-02-19T12:45:54.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:54.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-150323855362 2026-02-19T12:45:54.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:54.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:45:54.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-150323855362 2026-02-19T12:45:54.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:54.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855362 2026-02-19T12:45:54.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 150323855362' 2026-02-19T12:45:54.532 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 150323855362 2026-02-19T12:45:54.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:54.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018885 -lt 150323855362 2026-02-19T12:45:54.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:45:55.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:45:55.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:45:55.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855363 -lt 150323855362 2026-02-19T12:45:55.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:55.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705704 2026-02-19T12:45:55.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:55.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:45:55.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705704 2026-02-19T12:45:55.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:55.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705704 2026-02-19T12:45:55.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705704' 2026-02-19T12:45:55.856 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 38654705704 2026-02-19T12:45:55.856 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:45:56.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705704 -lt 38654705704 2026-02-19T12:45:56.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:45:56.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574884 2026-02-19T12:45:56.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:45:56.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:45:56.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574884 2026-02-19T12:45:56.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:45:56.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574884 2026-02-19T12:45:56.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574884' 2026-02-19T12:45:56.020 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 55834574884 2026-02-19T12:45:56.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:45:56.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574884 -lt 55834574884 2026-02-19T12:45:56.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:45:56.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:45:56.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:45:56.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:45:56.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:45:56.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:45:56.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:45:56.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:45:56.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:45:56.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:45:56.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:45:56.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:45:56.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:45:56.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:45:56.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:45:56.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:45:56.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:45:56.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:45:56.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:537: TEST_auto_repair_bluestore_basic: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:45:56.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:538: TEST_auto_repair_bluestore_basic: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-19T12:45:56.706 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:45:42.017+0000 7fd5f349f640 10 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=14/14 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep(periodic-regular,auto-repair)] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-19T12:45:56.706 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:45:42.017+0000 7fd5f349f640 10 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=14/14 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep(periodic-regular,auto-repair)] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-19T12:45:56.706 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:45:42.017+0000 7fd5f349f640 15 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=14/14 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair(periodic-regular,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish: 1 errors. 1 errors fixed 2026-02-19T12:45:56.706 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:45:42.017+0000 7fd5f349f640 20 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=14/14 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair(periodic-regular,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish All may be fixed 2026-02-19T12:45:56.706 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:45:42.017+0000 7fd5f349f640 19 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=14/14 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair(periodic-regular,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-19T12:45:56.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T12:45:56.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:45:56.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:45:56.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:45:56.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:45:56.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:45:56.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:45:56.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:45:56.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:45:56.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:45:56.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:45:56.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:45:56.819 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:45:56.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:45:56.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:45:56.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:45:56.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:45:56.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:45:56.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:45:56.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:45:56.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:45:56.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:45:56.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:45:56.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:45:56.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:45:56.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:45:56.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:45:56.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:45:56.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:45:56.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T12:45:56.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T12:45:56.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T12:45:56.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T12:45:56.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:45:56.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:45:56.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:45:56.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:45:56.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:45:56.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:45:56.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:45:56.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:45:56.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:45:56.842 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:45:56.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:45:56.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:45:56.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:45:56.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:45:56.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:45:56.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:45:56.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:45:56.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:45:56.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:45:56.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:45:56.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:45:56.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:45:56.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:45:56.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:45:56.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:45:56.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:45:56.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:45:56.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:45:56.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T12:45:56.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:45:56.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:45:56.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:45:56.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T12:45:56.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:45:56.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T12:45:56.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:45:56.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T12:45:56.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_bluestore_failed td/osd-scrub-repair 2026-02-19T12:45:56.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:598: TEST_auto_repair_bluestore_failed: local dir=td/osd-scrub-repair 2026-02-19T12:45:56.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:599: TEST_auto_repair_bluestore_failed: local poolname=testpool 2026-02-19T12:45:56.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:602: TEST_auto_repair_bluestore_failed: run_mon td/osd-scrub-repair a 2026-02-19T12:45:56.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T12:45:56.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T12:45:56.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T12:45:56.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T12:45:56.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T12:45:56.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T12:45:56.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T12:45:56.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:45:56.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:45:56.879 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:45:56.880 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:45:56.880 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:45:56.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:45:56.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T12:45:56.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T12:45:56.908 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T12:45:56.908 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:45:56.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:45:56.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T12:45:56.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T12:45:56.910 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:45:56.910 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:45:56.910 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:45:56.910 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:45:56.910 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:45:56.910 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:45:56.910 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:45:56.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:45:56.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T12:45:56.966 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T12:45:56.967 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:45:56.967 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:45:56.967 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T12:45:56.967 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T12:45:56.968 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:45:56.968 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:45:56.968 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:45:56.969 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:45:56.969 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:45:56.969 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:45:56.969 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:45:56.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:45:56.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T12:45:57.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:603: TEST_auto_repair_bluestore_failed: run_mgr td/osd-scrub-repair x 2026-02-19T12:45:57.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T12:45:57.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T12:45:57.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T12:45:57.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T12:45:57.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T12:45:57.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T12:45:57.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T12:45:57.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:45:57.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:45:57.118 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:45:57.118 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:45:57.119 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:45:57.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:45:57.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:45:57.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:45:57.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:606: TEST_auto_repair_bluestore_failed: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:45:57.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:607: TEST_auto_repair_bluestore_failed: seq 0 2 2026-02-19T12:45:57.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:607: TEST_auto_repair_bluestore_failed: for id in $(seq 0 2) 2026-02-19T12:45:57.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:608: TEST_auto_repair_bluestore_failed: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:45:57.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:45:57.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:45:57.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T12:45:57.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:45:57.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:45:57.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:45:57.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:45:57.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:45:57.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:45:57.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:45:57.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:45:57.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:45:57.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:45:57.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:45:57.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:45:57.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:45:57.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:45:57.145 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:45:57.145 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:45:57.145 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:45:57.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:45:57.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:45:57.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:45:57.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:45:57.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:45:57.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:45:57.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:45:57.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:45:57.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:45:57.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:45:57.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:45:57.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:45:57.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:45:57.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:45:57.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:45:57.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6e1cbeb4-c592-4d93-85ac-8a1d9eff3bbc 2026-02-19T12:45:57.147 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 6e1cbeb4-c592-4d93-85ac-8a1d9eff3bbc 2026-02-19T12:45:57.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 6e1cbeb4-c592-4d93-85ac-8a1d9eff3bbc' 2026-02-19T12:45:57.148 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:45:57.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAFBpdp9TFOCRAAWuiVNWjqjV6a6gznVvzqHw== 2026-02-19T12:45:57.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAFBpdp9TFOCRAAWuiVNWjqjV6a6gznVvzqHw=="}' 2026-02-19T12:45:57.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6e1cbeb4-c592-4d93-85ac-8a1d9eff3bbc -i td/osd-scrub-repair/0/new.json 2026-02-19T12:45:57.246 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:45:57.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T12:45:57.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQAFBpdp9TFOCRAAWuiVNWjqjV6a6gznVvzqHw== --osd-uuid 6e1cbeb4-c592-4d93-85ac-8a1d9eff3bbc 2026-02-19T12:45:57.272 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:57.265+0000 7ff726bc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:57.273 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:57.269+0000 7ff726bc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:57.274 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:57.269+0000 7ff726bc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:57.275 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:57.269+0000 7ff726bc1a40 -1 bdev(0x55f91d270c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:45:57.275 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:57.269+0000 7ff726bc1a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T12:45:57.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T12:45:57.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:45:57.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T12:45:57.965 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:45:57.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:45:58.162 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:45:58.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T12:45:58.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:45:58.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:45:58.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:45:58.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:45:58.189 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:58.185+0000 7f886dbb3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:58.191 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:58.185+0000 7f886dbb3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:58.192 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:58.185+0000 7f886dbb3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:45:58.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T12:45:58.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:45:58.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:45:58.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:45:58.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:45:58.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:45:58.321 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:45:58.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:45:58.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:45:58.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:45:58.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:45:58.570 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:58.565+0000 7f886dbb3a40 -1 Falling back to public interface 2026-02-19T12:45:59.197 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:45:59.193+0000 7f886dbb3a40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:45:59.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:45:59.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:45:59.482 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:45:59.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:45:59.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:45:59.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:45:59.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:46:00.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:46:00.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:00.641 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:46:00.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:46:00.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:00.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:46:00.799 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1129195715,v1:127.0.0.1:6803/1129195715] [v2:127.0.0.1:6804/1129195715,v1:127.0.0.1:6805/1129195715] exists,up 6e1cbeb4-c592-4d93-85ac-8a1d9eff3bbc 2026-02-19T12:46:00.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:46:00.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:46:00.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:46:00.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:607: TEST_auto_repair_bluestore_failed: for id in $(seq 0 2) 2026-02-19T12:46:00.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:608: TEST_auto_repair_bluestore_failed: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:46:00.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:46:00.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:46:00.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T12:46:00.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:46:00.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:46:00.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:46:00.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:46:00.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:46:00.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:46:00.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:46:00.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:46:00.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:46:00.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:46:00.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:46:00.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:46:00.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:46:00.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:46:00.803 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:46:00.803 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:46:00.803 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:46:00.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:00.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:00.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:46:00.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:46:00.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:46:00.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:46:00.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:46:00.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:46:00.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:46:00.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:46:00.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:46:00.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:46:00.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:46:00.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:46:00.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:46:00.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c5c7359a-7b97-4685-b184-8f1daa4d978f 2026-02-19T12:46:00.805 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 c5c7359a-7b97-4685-b184-8f1daa4d978f 2026-02-19T12:46:00.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 c5c7359a-7b97-4685-b184-8f1daa4d978f' 2026-02-19T12:46:00.806 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:46:00.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAIBpdpOXaVMBAA4vhTU/XYeoVR4nP+GkPlyw== 2026-02-19T12:46:00.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAIBpdpOXaVMBAA4vhTU/XYeoVR4nP+GkPlyw=="}' 2026-02-19T12:46:00.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c5c7359a-7b97-4685-b184-8f1daa4d978f -i td/osd-scrub-repair/1/new.json 2026-02-19T12:46:00.982 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:46:00.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T12:46:00.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQAIBpdpOXaVMBAA4vhTU/XYeoVR4nP+GkPlyw== --osd-uuid c5c7359a-7b97-4685-b184-8f1daa4d978f 2026-02-19T12:46:01.011 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:01.005+0000 7f1d253bfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:01.012 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:01.009+0000 7f1d253bfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:01.013 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:01.009+0000 7f1d253bfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:01.014 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:01.009+0000 7f1d253bfa40 -1 bdev(0x55fde8974c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:46:01.014 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:01.009+0000 7f1d253bfa40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T12:46:01.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T12:46:01.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:46:01.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T12:46:01.612 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:46:01.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:46:01.818 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:46:01.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T12:46:01.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:46:01.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:46:01.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:46:01.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:46:01.845 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:01.841+0000 7fa4f1b08a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:01.847 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:01.841+0000 7fa4f1b08a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:01.848 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:01.845+0000 7fa4f1b08a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:01.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T12:46:01.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:46:01.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:46:01.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:46:01.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:46:01.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:01.976 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:46:01.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:46:01.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:01.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:46:02.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:46:02.205 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:02.201+0000 7fa4f1b08a40 -1 Falling back to public interface 2026-02-19T12:46:02.801 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:02.797+0000 7fa4f1b08a40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:46:03.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:46:03.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:03.138 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:46:03.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:46:03.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:03.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:46:03.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:46:04.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:46:04.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:04.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:46:04.295 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:46:04.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:04.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:46:04.456 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2867757622,v1:127.0.0.1:6811/2867757622] [v2:127.0.0.1:6812/2867757622,v1:127.0.0.1:6813/2867757622] exists,up c5c7359a-7b97-4685-b184-8f1daa4d978f 2026-02-19T12:46:04.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:46:04.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:46:04.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:46:04.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:607: TEST_auto_repair_bluestore_failed: for id in $(seq 0 2) 2026-02-19T12:46:04.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:608: TEST_auto_repair_bluestore_failed: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:46:04.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:46:04.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:46:04.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T12:46:04.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:46:04.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:46:04.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:46:04.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:46:04.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:46:04.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:46:04.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:46:04.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:46:04.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:46:04.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:46:04.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:46:04.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:46:04.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:46:04.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:46:04.461 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:46:04.461 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:46:04.461 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:46:04.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:04.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:04.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:46:04.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:46:04.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:46:04.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:46:04.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:46:04.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:46:04.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:46:04.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:46:04.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:46:04.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:46:04.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:46:04.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:46:04.463 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:46:04.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6853ff9e-0a09-4d10-9955-d1cea87afcf4 2026-02-19T12:46:04.464 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 6853ff9e-0a09-4d10-9955-d1cea87afcf4 2026-02-19T12:46:04.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 6853ff9e-0a09-4d10-9955-d1cea87afcf4' 2026-02-19T12:46:04.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:46:04.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAMBpdpz7k3HBAA2GULimlL/aibuHe4pVli6w== 2026-02-19T12:46:04.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAMBpdpz7k3HBAA2GULimlL/aibuHe4pVli6w=="}' 2026-02-19T12:46:04.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6853ff9e-0a09-4d10-9955-d1cea87afcf4 -i td/osd-scrub-repair/2/new.json 2026-02-19T12:46:04.628 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:46:04.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T12:46:04.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQAMBpdpz7k3HBAA2GULimlL/aibuHe4pVli6w== --osd-uuid 6853ff9e-0a09-4d10-9955-d1cea87afcf4 2026-02-19T12:46:04.656 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:04.649+0000 7f0d91d8da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:04.657 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:04.653+0000 7f0d91d8da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:04.658 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:04.653+0000 7f0d91d8da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:04.659 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:04.653+0000 7f0d91d8da40 -1 bdev(0x5583b133ec00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T12:46:04.659 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:04.653+0000 7f0d91d8da40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T12:46:05.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T12:46:05.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:46:05.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T12:46:05.268 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T12:46:05.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:46:05.461 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T12:46:05.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T12:46:05.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:46:05.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:46:05.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:46:05.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:46:05.487 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:05.481+0000 7f4cf2729a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:05.489 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:05.485+0000 7f4cf2729a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:05.490 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:05.485+0000 7f4cf2729a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:05.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T12:46:05.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:46:05.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:46:05.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:46:05.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:46:05.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:05.621 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:46:05.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:46:05.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:05.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:46:05.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:46:05.853 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:05.849+0000 7f4cf2729a40 -1 Falling back to public interface 2026-02-19T12:46:06.445 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:06.441+0000 7f4cf2729a40 -1 osd.2 0 log_to_monitors true 2026-02-19T12:46:06.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:46:06.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:06.779 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:46:06.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:46:06.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:06.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:46:06.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:46:07.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:46:07.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:07.941 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:46:07.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:46:07.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:07.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:46:08.097 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/184971367,v1:127.0.0.1:6819/184971367] [v2:127.0.0.1:6820/184971367,v1:127.0.0.1:6821/184971367] exists,up 6853ff9e-0a09-4d10-9955-d1cea87afcf4 2026-02-19T12:46:08.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:46:08.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:46:08.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:46:08.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:611: TEST_auto_repair_bluestore_failed: create_pool testpool 1 1 2026-02-19T12:46:08.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create testpool 1 1 2026-02-19T12:46:08.342 INFO:tasks.workunit.client.0.trial144.stderr:pool 'testpool' created 2026-02-19T12:46:08.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:46:09.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:612: TEST_auto_repair_bluestore_failed: ceph osd pool set testpool size 2 2026-02-19T12:46:09.516 INFO:tasks.workunit.client.0.trial144.stderr:set pool 1 size to 2 2026-02-19T12:46:09.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:613: TEST_auto_repair_bluestore_failed: wait_for_clean 2026-02-19T12:46:09.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:46:09.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:46:09.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:46:09.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:46:09.527 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:46:09.527 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:46:09.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:46:09.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:46:09.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:46:09.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:46:09.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:46:09.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:46:09.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:46:09.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:46:09.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:46:09.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:46:09.795 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:46:09.795 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:46:09.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:46:09.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:09.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:46:09.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T12:46:09.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T12:46:09.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T12:46:09.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:09.868 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:46:09.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-19T12:46:09.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-19T12:46:09.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705667' 2026-02-19T12:46:09.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:09.939 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:46:10.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542146 2026-02-19T12:46:10.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542146 2026-02-19T12:46:10.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705667 2-60129542146' 2026-02-19T12:46:10.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:10.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T12:46:10.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:10.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:46:10.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T12:46:10.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:10.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T12:46:10.015 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T12:46:10.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T12:46:10.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:10.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T12:46:10.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:46:11.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:46:11.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:11.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T12:46:11.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:46:12.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:46:12.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:12.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836483 2026-02-19T12:46:12.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:12.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-19T12:46:12.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:12.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:46:12.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705667 2026-02-19T12:46:12.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:12.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705667 2026-02-19T12:46:12.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-19T12:46:12.514 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705667 2026-02-19T12:46:12.514 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:46:12.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705667 -lt 38654705667 2026-02-19T12:46:12.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:12.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542146 2026-02-19T12:46:12.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:12.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:46:12.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542146 2026-02-19T12:46:12.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:12.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542146 2026-02-19T12:46:12.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-19T12:46:12.687 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542146 2026-02-19T12:46:12.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:46:12.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542146 -lt 60129542146 2026-02-19T12:46:12.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:46:12.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:12.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:13.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:46:13.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:46:13.049 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:46:13.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:46:13.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:46:13.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:46:13.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:46:13.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:46:13.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:46:13.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:46:13.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:13.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:13.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:46:13.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:46:13.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:46:13.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:616: TEST_auto_repair_bluestore_failed: local payload=ABCDEF 2026-02-19T12:46:13.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:617: TEST_auto_repair_bluestore_failed: echo ABCDEF 2026-02-19T12:46:13.406 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:618: TEST_auto_repair_bluestore_failed: seq 1 10 2026-02-19T12:46:13.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:618: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-19T12:46:13.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:620: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj1 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:46:13.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:618: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-19T12:46:13.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:620: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj2 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:46:13.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:618: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-19T12:46:13.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:620: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj3 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:46:13.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:618: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-19T12:46:13.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:620: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj4 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:46:13.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:618: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-19T12:46:13.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:620: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj5 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:46:13.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:618: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-19T12:46:13.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:620: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj6 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:46:13.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:618: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-19T12:46:13.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:620: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj7 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:46:13.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:618: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-19T12:46:13.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:620: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj8 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:46:13.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:618: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-19T12:46:13.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:620: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj9 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:46:13.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:618: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-19T12:46:13.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:620: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj10 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:46:13.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:625: TEST_auto_repair_bluestore_failed: get_not_primary testpool SOMETHING 2026-02-19T12:46:13.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=testpool 2026-02-19T12:46:13.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:46:13.650 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary testpool SOMETHING 2026-02-19T12:46:13.650 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:46:13.651 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:46:13.651 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:46:13.651 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:46:13.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:46:13.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:46:13.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:46:13.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:625: TEST_auto_repair_bluestore_failed: objectstore_tool td/osd-scrub-repair 0 obj1 remove 2026-02-19T12:46:13.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:46:13.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:46:13.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:46:13.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:46:13.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 obj1 remove 2026-02-19T12:46:13.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:46:13.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:46:13.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:46:13.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:46:13.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:46:13.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:46:13.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:46:13.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:46:13.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:46:13.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:46:14.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:46:14.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 obj1 remove 2026-02-19T12:46:14.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:46:14.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:46:14.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:46:14.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:46:14.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:46:14.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj1 remove 2026-02-19T12:46:14.468 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-19T12:46:14.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:46:14.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:46:14.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:46:14.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:46:14.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:46:14.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:46:14.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:46:14.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:46:14.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:46:14.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:46:14.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:46:14.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:46:14.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:46:14.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:46:14.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:46:14.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:46:14.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:46:14.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:46:14.558 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:46:14.558 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:46:14.558 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:46:14.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:14.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:14.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:46:14.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:46:14.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:46:14.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:46:14.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:46:14.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:46:14.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:46:14.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:46:14.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:46:14.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:46:14.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:46:14.560 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:46:14.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:46:14.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:46:14.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:46:14.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:46:14.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:46:14.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:46:14.579 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:14.573+0000 7fb618dd9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:14.580 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:14.577+0000 7fb618dd9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:14.582 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:14.577+0000 7fb618dd9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:14.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:46:14.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:46:14.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:46:14.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:46:14.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:46:14.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:14.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:46:14.718 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:46:14.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:14.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:46:14.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:46:15.011 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:15.005+0000 7fb618dd9a40 -1 Falling back to public interface 2026-02-19T12:46:15.643 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:15.637+0000 7fb618dd9a40 -1 osd.0 18 log_to_monitors true 2026-02-19T12:46:15.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:46:15.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:15.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:46:15.879 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:46:15.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:15.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:46:16.036 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 22 up_thru 0 down_at 19 last_clean_interval [5,18) [v2:127.0.0.1:6802/3974524412,v1:127.0.0.1:6803/3974524412] [v2:127.0.0.1:6804/3974524412,v1:127.0.0.1:6805/3974524412] exists,up 6e1cbeb4-c592-4d93-85ac-8a1d9eff3bbc 2026-02-19T12:46:16.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:46:16.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:46:16.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:46:16.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:46:16.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:46:16.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:46:16.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:46:16.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:46:16.037 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:46:16.037 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:46:16.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:46:16.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:46:16.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:46:16.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:46:16.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:46:16.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:46:16.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:46:16.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:46:16.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:46:16.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:46:16.299 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:46:16.299 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:46:16.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:46:16.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:16.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:46:16.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280514 2026-02-19T12:46:16.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280514 2026-02-19T12:46:16.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-94489280514' 2026-02-19T12:46:16.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:16.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:46:16.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705669 2026-02-19T12:46:16.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705669 2026-02-19T12:46:16.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-94489280514 1-38654705669' 2026-02-19T12:46:16.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:16.443 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:46:16.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542148 2026-02-19T12:46:16.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542148 2026-02-19T12:46:16.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-94489280514 1-38654705669 2-60129542148' 2026-02-19T12:46:16.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:16.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-94489280514 2026-02-19T12:46:16.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:16.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:46:16.517 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-94489280514 2026-02-19T12:46:16.517 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:16.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280514 2026-02-19T12:46:16.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 94489280514' 2026-02-19T12:46:16.519 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 94489280514 2026-02-19T12:46:16.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:16.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 94489280514 2026-02-19T12:46:16.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:46:17.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:46:17.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:17.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280514 -lt 94489280514 2026-02-19T12:46:17.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:17.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705669 2026-02-19T12:46:17.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:17.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:46:17.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705669 2026-02-19T12:46:17.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:17.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705669 2026-02-19T12:46:17.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-19T12:46:17.854 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 38654705669 2026-02-19T12:46:17.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:46:18.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-19T12:46:18.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:18.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542148 2026-02-19T12:46:18.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:18.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:46:18.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542148 2026-02-19T12:46:18.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:18.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542148 2026-02-19T12:46:18.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-19T12:46:18.027 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542148 2026-02-19T12:46:18.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:46:18.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542149 -lt 60129542148 2026-02-19T12:46:18.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:46:18.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:18.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:18.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:46:18.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:46:18.397 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:46:18.398 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:46:18.398 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:46:18.398 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:46:18.398 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:46:18.398 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:46:18.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:46:18.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:46:18.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:18.552 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:18.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:46:18.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:46:18.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:46:18.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:627: TEST_auto_repair_bluestore_failed: get_not_primary testpool SOMETHING 2026-02-19T12:46:18.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=testpool 2026-02-19T12:46:18.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:46:18.743 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary testpool SOMETHING 2026-02-19T12:46:18.743 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:46:18.743 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:46:18.743 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:46:18.743 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:46:18.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:46:18.903 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:46:18.903 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:46:19.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:627: TEST_auto_repair_bluestore_failed: objectstore_tool td/osd-scrub-repair 0 obj2 remove 2026-02-19T12:46:19.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:46:19.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:46:19.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:46:19.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:46:19.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 obj2 remove 2026-02-19T12:46:19.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:46:19.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:46:19.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:46:19.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:46:19.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:46:19.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:46:19.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:46:19.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:46:19.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:46:19.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:46:19.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:46:19.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 obj2 remove 2026-02-19T12:46:19.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:46:19.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:46:19.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:46:19.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:46:19.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:46:19.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj2 remove 2026-02-19T12:46:19.535 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:104778fc:::obj2:head# 2026-02-19T12:46:19.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:46:19.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:46:19.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:46:19.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:46:19.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:46:19.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:46:19.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:46:19.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:46:19.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:46:19.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:46:19.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:46:19.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:46:19.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:46:19.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:46:19.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:46:19.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:46:19.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:46:19.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:46:19.627 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:46:19.627 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:46:19.627 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:46:19.628 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:19.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:19.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:46:19.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:46:19.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:46:19.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:46:19.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:46:19.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:46:19.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:46:19.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:46:19.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:46:19.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:46:19.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:46:19.630 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:46:19.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:46:19.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:46:19.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:46:19.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:46:19.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:46:19.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:46:19.653 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:19.648+0000 7f380d8e2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:19.654 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:19.648+0000 7f380d8e2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:19.655 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:19.648+0000 7f380d8e2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:19.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:46:19.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:46:19.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:46:19.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:46:19.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:46:19.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:19.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:46:19.785 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:46:19.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:19.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:46:19.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:46:20.046 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:20.040+0000 7f380d8e2a40 -1 Falling back to public interface 2026-02-19T12:46:20.698 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:20.692+0000 7f380d8e2a40 -1 osd.0 24 log_to_monitors true 2026-02-19T12:46:20.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:46:20.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:20.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:46:20.946 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:46:20.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:20.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:46:21.101 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 28 up_thru 0 down_at 25 last_clean_interval [22,24) [v2:127.0.0.1:6802/1612122482,v1:127.0.0.1:6803/1612122482] [v2:127.0.0.1:6804/1612122482,v1:127.0.0.1:6805/1612122482] exists,up 6e1cbeb4-c592-4d93-85ac-8a1d9eff3bbc 2026-02-19T12:46:21.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:46:21.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:46:21.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:46:21.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:46:21.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:46:21.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:46:21.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:46:21.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:46:21.103 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:46:21.103 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:46:21.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:46:21.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:46:21.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:46:21.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:46:21.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:46:21.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:46:21.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:46:21.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:46:21.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:46:21.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:46:21.378 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:46:21.378 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:46:21.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:46:21.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:21.379 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:46:21.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084290 2026-02-19T12:46:21.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084290 2026-02-19T12:46:21.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-120259084290' 2026-02-19T12:46:21.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:21.452 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:46:21.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705671 2026-02-19T12:46:21.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705671 2026-02-19T12:46:21.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-120259084290 1-38654705671' 2026-02-19T12:46:21.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:21.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:46:21.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542150 2026-02-19T12:46:21.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542150 2026-02-19T12:46:21.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-120259084290 1-38654705671 2-60129542150' 2026-02-19T12:46:21.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:21.598 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-120259084290 2026-02-19T12:46:21.598 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:21.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:46:21.600 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-120259084290 2026-02-19T12:46:21.600 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:21.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=120259084290 2026-02-19T12:46:21.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 120259084290' 2026-02-19T12:46:21.601 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 120259084290 2026-02-19T12:46:21.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:21.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084290 -lt 120259084290 2026-02-19T12:46:21.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:21.763 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705671 2026-02-19T12:46:21.764 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:21.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:46:21.765 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705671 2026-02-19T12:46:21.766 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:21.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705671 2026-02-19T12:46:21.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-19T12:46:21.767 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 38654705671 2026-02-19T12:46:21.768 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:46:21.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705671 -lt 38654705671 2026-02-19T12:46:21.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:21.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542150 2026-02-19T12:46:21.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:21.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:46:21.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542150 2026-02-19T12:46:21.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:21.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542150 2026-02-19T12:46:21.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-19T12:46:21.937 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542150 2026-02-19T12:46:21.937 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:46:22.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-19T12:46:22.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:46:22.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:22.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:22.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:46:22.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:46:22.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:46:22.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:46:22.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:46:22.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:46:22.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:46:22.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:46:22.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T12:46:22.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:46:22.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:22.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:22.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T12:46:22.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-19T12:46:22.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T12:46:22.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-19T12:46:22.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:46:22.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T12:46:22.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:46:22.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:46:22.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:46:22.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:46:22.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:46:22.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:46:22.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:46:22.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:46:22.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T12:46:22.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:46:22.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:22.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:23.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T12:46:23.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-19T12:46:23.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:46:23.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:46:23.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:46:23.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:46:23.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:46:23.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:46:23.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:46:23.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:46:23.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:46:23.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T12:46:23.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:46:23.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T12:46:23.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:46:23.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:46:23.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:46:23.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:46:23.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:46:23.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:46:23.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:46:23.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:46:23.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T12:46:23.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:46:23.657 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:23.657 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:23.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T12:46:23.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-19T12:46:23.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:46:23.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:46:23.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:46:23.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:46:23.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:46:23.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:46:23.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:46:24.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:46:24.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:46:24.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-19T12:46:24.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:46:24.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-19T12:46:24.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:46:24.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:46:24.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:46:24.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:46:24.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:46:24.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:46:24.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:46:24.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:46:24.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T12:46:24.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:46:24.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:24.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:24.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T12:46:24.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-19T12:46:24.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:46:24.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:46:24.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:46:24.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:46:24.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:46:24.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:46:24.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:46:24.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:46:24.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:46:24.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 13 )) 2026-02-19T12:46:24.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:46:24.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-19T12:46:25.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:46:25.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:46:25.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:46:25.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:46:25.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:46:25.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:46:25.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:46:25.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:46:25.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:46:25.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:46:25.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:25.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:26.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:46:26.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:46:26.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:46:26.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:628: TEST_auto_repair_bluestore_failed: get_primary testpool SOMETHING 2026-02-19T12:46:26.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:46:26.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:46:26.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:46:26.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:46:26.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:628: TEST_auto_repair_bluestore_failed: objectstore_tool td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-19T12:46:26.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:46:26.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:46:26.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:46:26.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:46:26.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-19T12:46:26.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:46:26.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:46:26.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:46:26.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:46:26.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:46:26.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:46:26.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:46:26.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:46:26.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:46:26.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:46:26.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:46:26.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-19T12:46:26.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:46:26.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:46:26.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:46:26.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:46:26.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:46:26.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj2 rm-attr _ 2026-02-19T12:46:26.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:46:26.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:46:26.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:46:26.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:46:26.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:46:26.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:46:26.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:46:26.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:46:26.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:46:26.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:46:26.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:46:26.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:46:26.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:46:26.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:46:26.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:46:26.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:46:26.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:46:26.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:46:26.901 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:46:26.901 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:46:26.901 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:46:26.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:26.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:26.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:46:26.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:46:26.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:46:26.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:46:26.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:46:26.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:46:26.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:46:26.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:46:26.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:46:26.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:46:26.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:46:26.903 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:46:26.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:46:26.903 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:46:26.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:46:26.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:46:26.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:46:26.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:46:26.915 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:26.908+0000 7fbb21e46a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:26.917 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:26.912+0000 7fbb21e46a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:26.918 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:26.912+0000 7fbb21e46a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:27.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:46:27.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:46:27.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:46:27.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:46:27.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:46:27.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:27.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:46:27.062 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:46:27.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:27.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:46:27.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:46:27.350 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:27.344+0000 7fbb21e46a40 -1 Falling back to public interface 2026-02-19T12:46:27.943 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:27.936+0000 7fbb21e46a40 -1 osd.1 29 log_to_monitors true 2026-02-19T12:46:28.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:46:28.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:28.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:46:28.220 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:46:28.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:28.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:46:28.379 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 33 up_thru 33 down_at 30 last_clean_interval [9,29) [v2:127.0.0.1:6810/3064492334,v1:127.0.0.1:6811/3064492334] [v2:127.0.0.1:6812/3064492334,v1:127.0.0.1:6813/3064492334] exists,up c5c7359a-7b97-4685-b184-8f1daa4d978f 2026-02-19T12:46:28.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:46:28.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:46:28.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:46:28.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:46:28.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:46:28.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:46:28.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:46:28.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:46:28.381 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:46:28.381 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:46:28.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:46:28.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:46:28.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:46:28.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:46:28.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:46:28.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:46:28.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:46:28.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:46:28.480 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:46:28.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:46:28.639 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:46:28.639 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:46:28.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:46:28.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:28.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:46:28.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084292 2026-02-19T12:46:28.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084292 2026-02-19T12:46:28.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-120259084292' 2026-02-19T12:46:28.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:28.714 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:46:28.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920770 2026-02-19T12:46:28.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920770 2026-02-19T12:46:28.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-120259084292 1-141733920770' 2026-02-19T12:46:28.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:28.788 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:46:28.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542153 2026-02-19T12:46:28.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542153 2026-02-19T12:46:28.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-120259084292 1-141733920770 2-60129542153' 2026-02-19T12:46:28.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:28.862 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-120259084292 2026-02-19T12:46:28.862 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:28.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:46:28.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-120259084292 2026-02-19T12:46:28.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:28.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=120259084292 2026-02-19T12:46:28.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 120259084292' 2026-02-19T12:46:28.866 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 120259084292 2026-02-19T12:46:28.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:29.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084291 -lt 120259084292 2026-02-19T12:46:29.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:46:30.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:46:30.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:30.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084292 -lt 120259084292 2026-02-19T12:46:30.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:30.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-141733920770 2026-02-19T12:46:30.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:30.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:46:30.193 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-141733920770 2026-02-19T12:46:30.193 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:30.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920770 2026-02-19T12:46:30.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 141733920770' 2026-02-19T12:46:30.195 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 141733920770 2026-02-19T12:46:30.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:46:30.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920770 -lt 141733920770 2026-02-19T12:46:30.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:30.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542153 2026-02-19T12:46:30.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:30.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:46:30.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542153 2026-02-19T12:46:30.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:30.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542153 2026-02-19T12:46:30.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-19T12:46:30.358 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542153 2026-02-19T12:46:30.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:46:30.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-19T12:46:30.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:46:30.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:30.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:30.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:46:30.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:46:30.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:46:30.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:46:30.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:46:30.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:46:30.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:46:30.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:46:30.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:46:30.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:46:30.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:30.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:31.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:46:31.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:46:31.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:46:31.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:630: TEST_auto_repair_bluestore_failed: get_pg testpool obj1 2026-02-19T12:46:31.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=testpool 2026-02-19T12:46:31.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=obj1 2026-02-19T12:46:31.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map testpool obj1 2026-02-19T12:46:31.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:46:31.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:630: TEST_auto_repair_bluestore_failed: local pgid=1.0 2026-02-19T12:46:31.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:631: TEST_auto_repair_bluestore_failed: get_primary testpool obj1 2026-02-19T12:46:31.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:46:31.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T12:46:31.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool obj1 2026-02-19T12:46:31.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:46:31.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:631: TEST_auto_repair_bluestore_failed: local primary=1 2026-02-19T12:46:31.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:632: TEST_auto_repair_bluestore_failed: get_last_scrub_stamp 1.0 2026-02-19T12:46:31.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:46:31.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:46:31.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:46:31.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:46:31.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:632: TEST_auto_repair_bluestore_failed: local last_scrub_stamp=2026-02-19T12:46:08.339996+0000 2026-02-19T12:46:31.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:633: TEST_auto_repair_bluestore_failed: ceph tell 1.0 schedule-deep-scrub 2026-02-19T12:46:31.610 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T12:46:31.611 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T12:46:31.611 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T12:46:31.611 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-08T07:58:21.610635+0000" 2026-02-19T12:46:31.611 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:46:31.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:636: TEST_auto_repair_bluestore_failed: wait_for_scrub 1.0 2026-02-19T12:46:08.339996+0000 2026-02-19T12:46:31.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T12:46:31.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:46:08.339996+0000 2026-02-19T12:46:31.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:46:31.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:46:31.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:46:31.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:46:31.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:46:31.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:46:31.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:46:31.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:46:31.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:46:08.339996+0000 '>' 2026-02-19T12:46:08.339996+0000 2026-02-19T12:46:31.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:46:32.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:46:32.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:46:32.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:46:32.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:46:32.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:46:32.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:46:32.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:46:32.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:46:08.339996+0000 '>' 2026-02-19T12:46:08.339996+0000 2026-02-19T12:46:32.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:46:33.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:46:33.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:46:33.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:46:33.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:46:33.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:46:33.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:46:33.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:46:34.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:46:32.888130+0000 '>' 2026-02-19T12:46:08.339996+0000 2026-02-19T12:46:34.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:46:34.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:637: TEST_auto_repair_bluestore_failed: wait_for_clean 2026-02-19T12:46:34.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:46:34.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:46:34.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:46:34.104 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:46:34.104 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:46:34.104 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:46:34.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:46:34.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:46:34.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:46:34.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:46:34.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:46:34.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:46:34.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:46:34.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:46:34.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:46:34.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:46:34.367 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:46:34.367 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:46:34.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:46:34.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:34.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:46:34.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084294 2026-02-19T12:46:34.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084294 2026-02-19T12:46:34.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-120259084294' 2026-02-19T12:46:34.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:34.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:46:34.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920772 2026-02-19T12:46:34.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920772 2026-02-19T12:46:34.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-120259084294 1-141733920772' 2026-02-19T12:46:34.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:34.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:46:34.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542155 2026-02-19T12:46:34.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542155 2026-02-19T12:46:34.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-120259084294 1-141733920772 2-60129542155' 2026-02-19T12:46:34.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:34.584 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-120259084294 2026-02-19T12:46:34.584 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:34.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:46:34.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-120259084294 2026-02-19T12:46:34.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:34.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=120259084294 2026-02-19T12:46:34.587 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 120259084294 2026-02-19T12:46:34.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 120259084294' 2026-02-19T12:46:34.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:34.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084293 -lt 120259084294 2026-02-19T12:46:34.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:46:35.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:46:35.756 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:35.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084294 -lt 120259084294 2026-02-19T12:46:35.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:35.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-141733920772 2026-02-19T12:46:35.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:35.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:46:35.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-141733920772 2026-02-19T12:46:35.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:35.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920772 2026-02-19T12:46:35.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 141733920772' 2026-02-19T12:46:35.917 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 141733920772 2026-02-19T12:46:35.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:46:36.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920772 -lt 141733920772 2026-02-19T12:46:36.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:36.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542155 2026-02-19T12:46:36.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:36.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:46:36.080 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542155 2026-02-19T12:46:36.080 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:36.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542155 2026-02-19T12:46:36.082 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542155 2026-02-19T12:46:36.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542155' 2026-02-19T12:46:36.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:46:36.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542155 -lt 60129542155 2026-02-19T12:46:36.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:46:36.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:36.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:36.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:46:36.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:46:36.441 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:46:36.441 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:46:36.441 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:46:36.441 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:46:36.441 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:46:36.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:46:36.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:46:36.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:46:36.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:36.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:36.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:46:36.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:46:36.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:46:36.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:638: TEST_auto_repair_bluestore_failed: flush_pg_stats 2026-02-19T12:46:36.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:46:36.785 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:46:36.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:46:36.944 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:46:36.944 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:46:36.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:46:36.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:36.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:46:37.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084296 2026-02-19T12:46:37.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084296 2026-02-19T12:46:37.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-120259084296' 2026-02-19T12:46:37.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:37.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:46:37.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920773 2026-02-19T12:46:37.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920773 2026-02-19T12:46:37.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-120259084296 1-141733920773' 2026-02-19T12:46:37.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:37.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:46:37.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542156 2026-02-19T12:46:37.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542156 2026-02-19T12:46:37.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-120259084296 1-141733920773 2-60129542156' 2026-02-19T12:46:37.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:37.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-120259084296 2026-02-19T12:46:37.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:37.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:46:37.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-120259084296 2026-02-19T12:46:37.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:37.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=120259084296 2026-02-19T12:46:37.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 120259084296' 2026-02-19T12:46:37.166 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 120259084296 2026-02-19T12:46:37.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:37.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084294 -lt 120259084296 2026-02-19T12:46:37.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:46:38.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:46:38.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:38.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084296 -lt 120259084296 2026-02-19T12:46:38.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:38.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-141733920773 2026-02-19T12:46:38.496 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:38.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:46:38.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-141733920773 2026-02-19T12:46:38.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:38.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920773 2026-02-19T12:46:38.499 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 141733920773 2026-02-19T12:46:38.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 141733920773' 2026-02-19T12:46:38.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:46:38.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920773 -lt 141733920773 2026-02-19T12:46:38.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:38.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542156 2026-02-19T12:46:38.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:38.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:46:38.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542156 2026-02-19T12:46:38.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:38.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542156 2026-02-19T12:46:38.679 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542156 2026-02-19T12:46:38.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542156' 2026-02-19T12:46:38.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:46:38.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542157 -lt 60129542156 2026-02-19T12:46:38.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:639: TEST_auto_repair_bluestore_failed: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-19T12:46:38.843 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:32.880+0000 7fbb035cb640 10 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep(periodic-regular,auto-repair)] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-19T12:46:38.843 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:32.880+0000 7fbb035cb640 10 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep(periodic-regular,auto-repair)] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-19T12:46:38.843 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:32.880+0000 7fbb035cb640 15 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair(periodic-regular,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish: 5 errors. 2 errors fixed 2026-02-19T12:46:38.843 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:32.880+0000 7fbb035cb640 20 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair(periodic-regular,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish: setting for deep-scrub-after-repair (5 errors. 2 errors fixed) 2026-02-19T12:46:38.843 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:32.880+0000 7fbb035cb640 19 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair(periodic-regular,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-19T12:46:38.843 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:33.888+0000 7fbb035cb640 10 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent(after-repair,check-repair)] scrubber: scrub_finish before flags: CHECK_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-19T12:46:38.844 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:33.888+0000 7fbb035cb640 10 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent(after-repair,check-repair)] scrubber: _scrub_finish info stats: valid m_is_repair: 0 2026-02-19T12:46:38.844 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:33.888+0000 7fbb035cb640 19 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent(after-repair,check-repair)] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-19T12:46:38.844 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:33.888+0000 7fbb035cb640 10 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent+failed_repair(after-repair)] scrubber: scrub_finish 3 error(s) still present after re-scrub 2026-02-19T12:46:38.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:640: TEST_auto_repair_bluestore_failed: grep -q 'scrub_finish.*still present after re-scrub' td/osd-scrub-repair/osd.1.log 2026-02-19T12:46:38.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:641: TEST_auto_repair_bluestore_failed: ceph pg dump pgs 2026-02-19T12:46:39.000 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:46:39.000 INFO:tasks.workunit.client.0.trial144.stdout:1.0 10 0 0 0 0 63 0 0 10 0 10 active+clean+inconsistent+failed_repair 2026-02-19T12:46:33.892101+0000 18'10 34:111 [1,0] 1 [1,0] 1 18'10 2026-02-19T12:46:33.892042+0000 18'10 2026-02-19T12:46:33.892042+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:46:33.892042+0000 10 0 2026-02-19T12:46:39.000 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:46:39.000 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:46:39.000 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:46:39.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:642: TEST_auto_repair_bluestore_failed: ceph pg dump pgs 2026-02-19T12:46:39.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:642: TEST_auto_repair_bluestore_failed: grep -q '^1.0.*+failed_repair' 2026-02-19T12:46:39.167 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:46:39.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:646: TEST_auto_repair_bluestore_failed: get_not_primary testpool obj1 2026-02-19T12:46:39.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=testpool 2026-02-19T12:46:39.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=obj1 2026-02-19T12:46:39.180 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary testpool obj1 2026-02-19T12:46:39.180 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:46:39.180 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T12:46:39.180 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool obj1 2026-02-19T12:46:39.180 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:46:39.339 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:46:39.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map testpool obj1 2026-02-19T12:46:39.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:46:39.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:646: TEST_auto_repair_bluestore_failed: objectstore_tool td/osd-scrub-repair 0 obj1 list-attrs 2026-02-19T12:46:39.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:46:39.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:46:39.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:46:39.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:46:39.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 obj1 list-attrs 2026-02-19T12:46:39.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:46:39.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:46:39.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:46:39.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:46:39.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:46:39.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:46:39.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:46:39.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:46:39.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:46:39.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:46:39.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:46:39.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 obj1 list-attrs 2026-02-19T12:46:39.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:46:39.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:46:39.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:46:39.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:46:39.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:46:39.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj1 list-attrs 2026-02-19T12:46:39.879 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:46:39.879 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:46:39.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:46:39.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:46:39.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:46:39.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:46:39.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:46:39.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:46:39.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:46:39.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:46:39.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:46:39.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:46:39.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:46:39.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:46:39.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:46:39.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:46:39.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:46:39.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:46:39.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:46:39.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:46:39.980 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:46:39.980 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:46:39.980 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:46:39.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:39.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:39.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:46:39.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:46:39.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:46:39.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:46:39.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:46:39.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:46:39.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:46:39.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:46:39.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:46:39.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:46:39.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:46:39.982 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:46:39.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:46:39.983 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:46:39.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:46:39.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:46:39.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:46:39.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:46:39.998 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:39.992+0000 7fdddc919a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:39.999 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:39.992+0000 7fdddc919a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:40.000 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:39.996+0000 7fdddc919a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:40.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:46:40.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:46:40.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:46:40.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:46:40.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:46:40.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:40.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:46:40.143 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:46:40.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:40.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:46:40.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:46:40.413 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:40.408+0000 7fdddc919a40 -1 Falling back to public interface 2026-02-19T12:46:41.015 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:41.008+0000 7fdddc919a40 -1 osd.0 35 log_to_monitors true 2026-02-19T12:46:41.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:46:41.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:41.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:46:41.297 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:46:41.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:41.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:46:41.455 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 39 up_thru 30 down_at 35 last_clean_interval [28,35) [v2:127.0.0.1:6802/1671622114,v1:127.0.0.1:6803/1671622114] [v2:127.0.0.1:6804/1671622114,v1:127.0.0.1:6805/1671622114] exists,up 6e1cbeb4-c592-4d93-85ac-8a1d9eff3bbc 2026-02-19T12:46:41.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:46:41.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:46:41.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:46:41.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:46:41.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:46:41.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:46:41.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:46:41.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:46:41.456 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:46:41.456 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:46:41.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:46:41.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:46:41.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:46:41.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:46:41.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:46:41.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:46:41.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:46:41.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:46:41.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:46:41.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:46:41.728 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:46:41.729 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:46:41.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:46:41.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:41.729 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:46:41.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724546 2026-02-19T12:46:41.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724546 2026-02-19T12:46:41.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724546' 2026-02-19T12:46:41.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:41.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:46:41.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920775 2026-02-19T12:46:41.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920775 2026-02-19T12:46:41.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724546 1-141733920775' 2026-02-19T12:46:41.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:41.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:46:41.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542158 2026-02-19T12:46:41.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542158 2026-02-19T12:46:41.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724546 1-141733920775 2-60129542158' 2026-02-19T12:46:41.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:41.946 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-167503724546 2026-02-19T12:46:41.946 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:41.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:46:41.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-167503724546 2026-02-19T12:46:41.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:41.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724546 2026-02-19T12:46:41.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 167503724546' 2026-02-19T12:46:41.950 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 167503724546 2026-02-19T12:46:41.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:42.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 167503724546 2026-02-19T12:46:42.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:46:43.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:46:43.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:43.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 167503724546 2026-02-19T12:46:43.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:46:44.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:46:44.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:44.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724546 -lt 167503724546 2026-02-19T12:46:44.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:44.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-141733920775 2026-02-19T12:46:44.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:44.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:46:44.444 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-141733920775 2026-02-19T12:46:44.444 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:44.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920775 2026-02-19T12:46:44.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 141733920775' 2026-02-19T12:46:44.446 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 141733920775 2026-02-19T12:46:44.446 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:46:44.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920776 -lt 141733920775 2026-02-19T12:46:44.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:44.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542158 2026-02-19T12:46:44.615 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:44.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:46:44.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542158 2026-02-19T12:46:44.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:44.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542158 2026-02-19T12:46:44.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542158' 2026-02-19T12:46:44.618 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542158 2026-02-19T12:46:44.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:46:44.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542159 -lt 60129542158 2026-02-19T12:46:44.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:46:44.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:44.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:44.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:46:44.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:46:44.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:46:44.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:46:44.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:46:44.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:46:44.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:46:44.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:46:45.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:46:45.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:46:45.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:45.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:45.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:46:45.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:46:45.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:46:45.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:647: TEST_auto_repair_bluestore_failed: rados --pool testpool get obj1 td/osd-scrub-repair/COPY 2026-02-19T12:46:45.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:648: TEST_auto_repair_bluestore_failed: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:46:45.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:649: TEST_auto_repair_bluestore_failed: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-19T12:46:45.338 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:32.880+0000 7fbb035cb640 10 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep(periodic-regular,auto-repair)] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-19T12:46:45.338 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:32.880+0000 7fbb035cb640 10 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep(periodic-regular,auto-repair)] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-19T12:46:45.338 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:32.880+0000 7fbb035cb640 15 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair(periodic-regular,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish: 5 errors. 2 errors fixed 2026-02-19T12:46:45.339 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:32.880+0000 7fbb035cb640 20 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair(periodic-regular,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish: setting for deep-scrub-after-repair (5 errors. 2 errors fixed) 2026-02-19T12:46:45.339 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:32.880+0000 7fbb035cb640 19 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair(periodic-regular,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-19T12:46:45.339 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:33.888+0000 7fbb035cb640 10 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent(after-repair,check-repair)] scrubber: scrub_finish before flags: CHECK_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-19T12:46:45.339 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:33.888+0000 7fbb035cb640 10 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent(after-repair,check-repair)] scrubber: _scrub_finish info stats: valid m_is_repair: 0 2026-02-19T12:46:45.339 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:33.888+0000 7fbb035cb640 19 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent(after-repair,check-repair)] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-19T12:46:45.339 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:33.888+0000 7fbb035cb640 10 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent+failed_repair(after-repair)] scrubber: scrub_finish 3 error(s) still present after re-scrub 2026-02-19T12:46:45.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:652: TEST_auto_repair_bluestore_failed: get_primary testpool SOMETHING 2026-02-19T12:46:45.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:46:45.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:46:45.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:46:45.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:46:45.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:652: TEST_auto_repair_bluestore_failed: objectstore_tool td/osd-scrub-repair 1 obj2 remove 2026-02-19T12:46:45.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:46:45.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:46:45.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:46:45.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:46:45.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 obj2 remove 2026-02-19T12:46:45.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:46:45.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:46:45.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:46:45.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:46:45.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:46:45.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:46:45.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:46:45.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:46:45.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:46:45.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:46:45.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:46:45.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 obj2 remove 2026-02-19T12:46:45.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:46:45.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:46:45.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:46:45.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:46:45.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:46:45.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj2 remove 2026-02-19T12:46:45.971 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:104778fc:::obj2:head#, (61) No data available 2026-02-19T12:46:45.972 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:104778fc:::obj2:head# 2026-02-19T12:46:46.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:46:46.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:46:46.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:46:46.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:46:46.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:46:46.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:46:46.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:46:46.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:46:46.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:46:46.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:46:46.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:46:46.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:46:46.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:46:46.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:46:46.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:46:46.072 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:46:46.072 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:46:46.072 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:46:46.073 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:46:46.073 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:46:46.073 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:46:46.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:46.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:46.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:46:46.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:46:46.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:46:46.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:46:46.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:46:46.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:46:46.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:46:46.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:46:46.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:46:46.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:46:46.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:46:46.075 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:46:46.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:46:46.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:46:46.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:46:46.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:46:46.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:46:46.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:46:46.092 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:46.088+0000 7fed69121a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:46.093 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:46.088+0000 7fed69121a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:46.094 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:46.088+0000 7fed69121a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:46.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:46:46.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:46:46.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:46:46.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:46:46.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:46:46.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:46.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:46:46.233 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:46:46.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:46.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:46:46.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:46:46.474 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:46.468+0000 7fed69121a40 -1 Falling back to public interface 2026-02-19T12:46:47.027 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:47.020+0000 7fed69121a40 -1 osd.1 41 log_to_monitors true 2026-02-19T12:46:47.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:46:47.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:46:47.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:46:47.393 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:46:47.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:46:47.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:46:47.549 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 44 up_thru 44 down_at 41 last_clean_interval [33,41) [v2:127.0.0.1:6810/2111244804,v1:127.0.0.1:6811/2111244804] [v2:127.0.0.1:6812/2111244804,v1:127.0.0.1:6813/2111244804] exists,up c5c7359a-7b97-4685-b184-8f1daa4d978f 2026-02-19T12:46:47.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:46:47.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:46:47.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:46:47.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:46:47.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:46:47.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:46:47.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:46:47.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:46:47.551 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:46:47.551 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:46:47.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:46:47.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:46:47.552 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:46:47.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:46:47.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:46:47.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:46:47.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:46:47.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:46:47.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:46:47.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:46:47.814 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:46:47.814 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:46:47.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:46:47.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:47.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:46:47.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724548 2026-02-19T12:46:47.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724548 2026-02-19T12:46:47.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724548' 2026-02-19T12:46:47.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:47.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:46:47.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561026 2026-02-19T12:46:47.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561026 2026-02-19T12:46:47.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724548 1-188978561026' 2026-02-19T12:46:47.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:47.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:46:48.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542161 2026-02-19T12:46:48.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542161 2026-02-19T12:46:48.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724548 1-188978561026 2-60129542161' 2026-02-19T12:46:48.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:48.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-167503724548 2026-02-19T12:46:48.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:48.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:46:48.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-167503724548 2026-02-19T12:46:48.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:48.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724548 2026-02-19T12:46:48.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 167503724548' 2026-02-19T12:46:48.034 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 167503724548 2026-02-19T12:46:48.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:48.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724547 -lt 167503724548 2026-02-19T12:46:48.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:46:49.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:46:49.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:49.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724547 -lt 167503724548 2026-02-19T12:46:49.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:46:50.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:46:50.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:50.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724548 -lt 167503724548 2026-02-19T12:46:50.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:50.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-188978561026 2026-02-19T12:46:50.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:50.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:46:50.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-188978561026 2026-02-19T12:46:50.525 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:50.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=188978561026 2026-02-19T12:46:50.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 188978561026' 2026-02-19T12:46:50.526 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 188978561026 2026-02-19T12:46:50.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:46:50.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 188978561026 -lt 188978561026 2026-02-19T12:46:50.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:50.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542161 2026-02-19T12:46:50.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:50.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:46:50.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542161 2026-02-19T12:46:50.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:50.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542161 2026-02-19T12:46:50.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542161' 2026-02-19T12:46:50.701 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542161 2026-02-19T12:46:50.701 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:46:50.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542161 -lt 60129542161 2026-02-19T12:46:50.858 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:46:50.858 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:50.858 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:51.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:46:51.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:46:51.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:46:51.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:46:51.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:46:51.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:46:51.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:46:51.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:46:51.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:46:51.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:46:51.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:46:51.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:46:51.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:46:51.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:46:51.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:46:51.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:653: TEST_auto_repair_bluestore_failed: repair 1.0 2026-02-19T12:46:51.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=1.0 2026-02-19T12:46:51.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 1.0 2026-02-19T12:46:51.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:46:51.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:46:51.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:46:51.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:46:51.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:46:33.892042+0000 2026-02-19T12:46:51.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 1.0 2026-02-19T12:46:51.703 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.0 on osd.1 to repair 2026-02-19T12:46:51.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 1.0 2026-02-19T12:46:33.892042+0000 2026-02-19T12:46:51.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T12:46:51.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:46:33.892042+0000 2026-02-19T12:46:51.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:46:51.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:46:51.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:46:51.714 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:46:51.714 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:46:51.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:46:51.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:46:51.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:46:51.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:46:33.892042+0000 '>' 2026-02-19T12:46:33.892042+0000 2026-02-19T12:46:51.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:46:52.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:46:52.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:46:52.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:46:52.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:46:52.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:46:52.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:46:52.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:46:53.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:46:33.892042+0000 '>' 2026-02-19T12:46:33.892042+0000 2026-02-19T12:46:53.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:46:54.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:46:54.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:46:54.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:46:54.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:46:54.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:46:54.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:46:54.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:46:54.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:46:51.979445+0000 '>' 2026-02-19T12:46:33.892042+0000 2026-02-19T12:46:54.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:46:54.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:654: TEST_auto_repair_bluestore_failed: sleep 2 2026-02-19T12:46:56.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:656: TEST_auto_repair_bluestore_failed: flush_pg_stats 2026-02-19T12:46:56.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:46:56.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:46:56.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:46:56.367 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:46:56.367 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:46:56.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:46:56.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:56.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:46:56.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724551 2026-02-19T12:46:56.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724551 2026-02-19T12:46:56.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724551' 2026-02-19T12:46:56.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:56.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:46:56.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561028 2026-02-19T12:46:56.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561028 2026-02-19T12:46:56.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724551 1-188978561028' 2026-02-19T12:46:56.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:46:56.514 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:46:56.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542163 2026-02-19T12:46:56.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542163 2026-02-19T12:46:56.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724551 1-188978561028 2-60129542163' 2026-02-19T12:46:56.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:56.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-167503724551 2026-02-19T12:46:56.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:56.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:46:56.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-167503724551 2026-02-19T12:46:56.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:56.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724551 2026-02-19T12:46:56.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 167503724551' 2026-02-19T12:46:56.590 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 167503724551 2026-02-19T12:46:56.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:56.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724549 -lt 167503724551 2026-02-19T12:46:56.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:46:57.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:46:57.759 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:46:57.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724551 -lt 167503724551 2026-02-19T12:46:57.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:57.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-188978561028 2026-02-19T12:46:57.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:57.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:46:57.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-188978561028 2026-02-19T12:46:57.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:57.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=188978561028 2026-02-19T12:46:57.920 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 188978561028 2026-02-19T12:46:57.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 188978561028' 2026-02-19T12:46:57.921 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:46:58.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 188978561029 -lt 188978561028 2026-02-19T12:46:58.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:46:58.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542163 2026-02-19T12:46:58.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:46:58.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:46:58.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542163 2026-02-19T12:46:58.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:46:58.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542163 2026-02-19T12:46:58.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542163' 2026-02-19T12:46:58.081 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542163 2026-02-19T12:46:58.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:46:58.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542164 -lt 60129542163 2026-02-19T12:46:58.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:657: TEST_auto_repair_bluestore_failed: ceph pg dump pgs 2026-02-19T12:46:58.392 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:46:58.392 INFO:tasks.workunit.client.0.trial144.stdout:1.0 9 0 0 0 0 63 0 0 10 0 10 active+clean 2026-02-19T12:46:51.979497+0000 18'10 45:153 [1,0] 1 [1,0] 1 18'10 2026-02-19T12:46:51.979445+0000 18'10 2026-02-19T12:46:51.979445+0000 0 0 periodic scrub scheduled @ 2026-02-20T12:46:51.979445+0000 9 0 2026-02-19T12:46:58.392 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:46:58.392 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:46:58.393 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:46:58.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:658: TEST_auto_repair_bluestore_failed: ceph pg dump pgs 2026-02-19T12:46:58.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:658: TEST_auto_repair_bluestore_failed: grep -q -e '^1.0.* active+clean ' -e '^1.0.* active+clean+wait ' 2026-02-19T12:46:58.562 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:46:58.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:659: TEST_auto_repair_bluestore_failed: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-19T12:46:58.576 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:32.880+0000 7fbb035cb640 10 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep(periodic-regular,auto-repair)] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-19T12:46:58.576 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:32.880+0000 7fbb035cb640 10 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep(periodic-regular,auto-repair)] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-19T12:46:58.576 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:32.880+0000 7fbb035cb640 15 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair(periodic-regular,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish: 5 errors. 2 errors fixed 2026-02-19T12:46:58.576 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:32.880+0000 7fbb035cb640 20 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair(periodic-regular,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish: setting for deep-scrub-after-repair (5 errors. 2 errors fixed) 2026-02-19T12:46:58.576 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:32.880+0000 7fbb035cb640 19 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair(periodic-regular,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-19T12:46:58.576 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:33.888+0000 7fbb035cb640 10 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent(after-repair,check-repair)] scrubber: scrub_finish before flags: CHECK_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-19T12:46:58.576 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:33.888+0000 7fbb035cb640 10 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent(after-repair,check-repair)] scrubber: _scrub_finish info stats: valid m_is_repair: 0 2026-02-19T12:46:58.576 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:33.888+0000 7fbb035cb640 19 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent(after-repair,check-repair)] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-19T12:46:58.577 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:33.888+0000 7fbb035cb640 10 osd.1 pg_epoch: 34 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=33/34 n=10 ec=15/15 lis/c=33/33 les/c/f=34/34/0 sis=33) [1,0] r=0 lpr=33 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent+failed_repair(after-repair)] scrubber: scrub_finish 3 error(s) still present after re-scrub 2026-02-19T12:46:58.577 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:51.972+0000 7fed4689e640 10 osd.1 pg_epoch: 45 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=44/45 n=10 ec=15/15 lis/c=44/44 les/c/f=45/45/0 sis=44) [1,0] r=0 lpr=44 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+repair(must-repair,auto-repair)] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: repair. deep_scrub_on_error: 0 2026-02-19T12:46:58.577 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:51.972+0000 7fed4689e640 10 osd.1 pg_epoch: 45 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=44/45 n=10 ec=15/15 lis/c=44/44 les/c/f=45/45/0 sis=44) [1,0] r=0 lpr=44 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+repair(must-repair,auto-repair)] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-19T12:46:58.577 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:51.972+0000 7fed4689e640 15 osd.1 pg_epoch: 45 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=44/45 n=9 ec=15/15 lis/c=44/44 les/c/f=45/45/0 sis=44) [1,0] r=0 lpr=44 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+repair(must-repair,auto-repair)] scrubber: scrub_finish: 1 errors. 1 errors fixed 2026-02-19T12:46:58.577 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:51.972+0000 7fed4689e640 20 osd.1 pg_epoch: 45 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=44/45 n=9 ec=15/15 lis/c=44/44 les/c/f=45/45/0 sis=44) [1,0] r=0 lpr=44 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+repair(must-repair,auto-repair)] scrubber: scrub_finish All may be fixed 2026-02-19T12:46:58.577 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:46:51.972+0000 7fed4689e640 19 osd.1 pg_epoch: 45 pg[1.0( v 18'10 (0'0,18'10] local-lis/les=44/45 n=9 ec=15/15 lis/c=44/44 les/c/f=45/45/0 sis=44) [1,0] r=0 lpr=44 crt=18'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+repair(must-repair,auto-repair)] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-19T12:46:58.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T12:46:58.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:46:58.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:46:58.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:46:58.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:46:58.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:46:58.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:46:58.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:46:58.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:46:58.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:46:58.688 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:46:58.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:46:58.689 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:46:58.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:46:58.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:46:58.691 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:46:58.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:46:58.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:46:58.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:46:58.693 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:46:58.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:46:58.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:46:58.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:46:58.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:46:58.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:46:58.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:46:58.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:46:58.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:46:58.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:46:58.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T12:46:58.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T12:46:58.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T12:46:58.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T12:46:58.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:46:58.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:46:58.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:46:58.713 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:46:58.713 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:46:58.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:46:58.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:46:58.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:46:58.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:46:58.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:46:58.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:46:58.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:46:58.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:46:58.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:46:58.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:46:58.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:46:58.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:46:58.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:46:58.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:46:58.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:46:58.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:46:58.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:46:58.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:46:58.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:46:58.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:46:58.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:46:58.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:46:58.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:46:58.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T12:46:58.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:46:58.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:46:58.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:46:58.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T12:46:58.729 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:46:58.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T12:46:58.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:46:58.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T12:46:58.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_bluestore_failed_norecov td/osd-scrub-repair 2026-02-19T12:46:58.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:663: TEST_auto_repair_bluestore_failed_norecov: local dir=td/osd-scrub-repair 2026-02-19T12:46:58.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:664: TEST_auto_repair_bluestore_failed_norecov: local poolname=testpool 2026-02-19T12:46:58.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:667: TEST_auto_repair_bluestore_failed_norecov: run_mon td/osd-scrub-repair a 2026-02-19T12:46:58.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T12:46:58.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T12:46:58.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T12:46:58.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T12:46:58.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T12:46:58.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T12:46:58.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T12:46:58.752 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:46:58.752 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:46:58.752 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:46:58.752 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:46:58.752 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:46:58.752 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:58.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T12:46:58.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T12:46:58.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T12:46:58.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:46:58.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:46:58.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T12:46:58.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T12:46:58.781 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:46:58.781 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:46:58.781 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:46:58.782 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:46:58.782 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:46:58.782 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:46:58.782 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:46:58.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:46:58.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T12:46:58.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T12:46:58.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:46:58.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:46:58.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T12:46:58.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T12:46:58.839 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:46:58.839 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:46:58.839 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:46:58.839 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:46:58.839 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:46:58.839 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:46:58.840 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:46:58.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:46:58.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T12:46:58.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:668: TEST_auto_repair_bluestore_failed_norecov: run_mgr td/osd-scrub-repair x 2026-02-19T12:46:58.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T12:46:58.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T12:46:58.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T12:46:58.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T12:46:58.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T12:46:58.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T12:46:58.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T12:46:58.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:46:58.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:46:58.990 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:46:58.990 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:46:58.990 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:46:58.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:58.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:46:58.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:46:59.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:671: TEST_auto_repair_bluestore_failed_norecov: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:46:59.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:672: TEST_auto_repair_bluestore_failed_norecov: seq 0 2 2026-02-19T12:46:59.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:672: TEST_auto_repair_bluestore_failed_norecov: for id in $(seq 0 2) 2026-02-19T12:46:59.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:673: TEST_auto_repair_bluestore_failed_norecov: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:46:59.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:46:59.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:46:59.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T12:46:59.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:46:59.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:46:59.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:46:59.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:46:59.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:46:59.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:46:59.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:46:59.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:46:59.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:46:59.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:46:59.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:46:59.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:46:59.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:46:59.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:46:59.021 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:46:59.021 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:46:59.021 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:46:59.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:59.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:46:59.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:46:59.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:46:59.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:46:59.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:46:59.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:46:59.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:46:59.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:46:59.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:46:59.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:46:59.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:46:59.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:46:59.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:46:59.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:46:59.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=09bf220f-c5cd-4ce9-a942-3f1fbf430b5b 2026-02-19T12:46:59.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 09bf220f-c5cd-4ce9-a942-3f1fbf430b5b' 2026-02-19T12:46:59.023 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 09bf220f-c5cd-4ce9-a942-3f1fbf430b5b 2026-02-19T12:46:59.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:46:59.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBDBpdpOpd/ARAAp7mlb0E4jq3dSSA5ymGGXg== 2026-02-19T12:46:59.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBDBpdpOpd/ARAAp7mlb0E4jq3dSSA5ymGGXg=="}' 2026-02-19T12:46:59.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 09bf220f-c5cd-4ce9-a942-3f1fbf430b5b -i td/osd-scrub-repair/0/new.json 2026-02-19T12:46:59.113 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:46:59.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T12:46:59.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQBDBpdpOpd/ARAAp7mlb0E4jq3dSSA5ymGGXg== --osd-uuid 09bf220f-c5cd-4ce9-a942-3f1fbf430b5b 2026-02-19T12:46:59.138 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:59.136+0000 7f49bc45da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:59.140 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:59.140+0000 7f49bc45da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:59.141 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:59.140+0000 7f49bc45da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:46:59.141 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:59.140+0000 7f49bc45da40 -1 bdev(0x560b4f024c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:46:59.141 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:46:59.140+0000 7f49bc45da40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T12:46:59.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T12:46:59.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:46:59.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T12:46:59.848 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:46:59.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:47:00.051 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:47:00.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T12:47:00.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:47:00.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:47:00.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:47:00.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:47:00.068 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:00.068+0000 7fbc06688a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:00.070 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:00.068+0000 7fbc06688a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:00.071 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:00.068+0000 7fbc06688a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:00.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T12:47:00.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:47:00.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:47:00.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:47:00.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:47:00.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:00.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:47:00.236 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:47:00.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:00.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:47:00.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:47:00.521 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:00.520+0000 7fbc06688a40 -1 Falling back to public interface 2026-02-19T12:47:01.143 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:01.140+0000 7fbc06688a40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:47:01.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:47:01.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:01.392 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:47:01.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:47:01.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:01.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:47:01.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:47:02.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:47:02.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:02.555 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:47:02.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:47:02.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:02.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:47:02.713 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/307550089,v1:127.0.0.1:6803/307550089] [v2:127.0.0.1:6804/307550089,v1:127.0.0.1:6805/307550089] exists,up 09bf220f-c5cd-4ce9-a942-3f1fbf430b5b 2026-02-19T12:47:02.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:47:02.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:47:02.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:47:02.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:672: TEST_auto_repair_bluestore_failed_norecov: for id in $(seq 0 2) 2026-02-19T12:47:02.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:673: TEST_auto_repair_bluestore_failed_norecov: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:47:02.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:47:02.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:47:02.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T12:47:02.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:47:02.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:47:02.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:47:02.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:47:02.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:47:02.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:47:02.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:47:02.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:47:02.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:47:02.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:47:02.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:47:02.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:47:02.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:47:02.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:47:02.719 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:47:02.719 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:47:02.719 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:47:02.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:02.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:02.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:47:02.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:47:02.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:47:02.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:47:02.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:47:02.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:47:02.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:47:02.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:47:02.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:47:02.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:47:02.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:47:02.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:47:02.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:47:02.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3b4b8f5e-82c1-403b-b67f-be60d495916c 2026-02-19T12:47:02.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 3b4b8f5e-82c1-403b-b67f-be60d495916c' 2026-02-19T12:47:02.721 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 3b4b8f5e-82c1-403b-b67f-be60d495916c 2026-02-19T12:47:02.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:47:02.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBGBpdpwdGGKxAAMbYYfxxAVoJx4L6F7HeOhQ== 2026-02-19T12:47:02.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBGBpdpwdGGKxAAMbYYfxxAVoJx4L6F7HeOhQ=="}' 2026-02-19T12:47:02.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3b4b8f5e-82c1-403b-b67f-be60d495916c -i td/osd-scrub-repair/1/new.json 2026-02-19T12:47:02.884 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:47:02.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T12:47:02.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQBGBpdpwdGGKxAAMbYYfxxAVoJx4L6F7HeOhQ== --osd-uuid 3b4b8f5e-82c1-403b-b67f-be60d495916c 2026-02-19T12:47:02.912 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:02.912+0000 7f6d384baa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:02.913 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:02.912+0000 7f6d384baa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:02.914 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:02.912+0000 7f6d384baa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:02.915 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:02.912+0000 7f6d384baa40 -1 bdev(0x55b98b026c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:47:02.915 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:02.912+0000 7f6d384baa40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T12:47:03.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T12:47:03.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:47:03.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T12:47:03.612 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:47:03.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:47:03.811 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:47:03.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T12:47:03.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:47:03.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:47:03.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:47:03.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:47:03.838 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:03.836+0000 7fd11601ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:03.839 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:03.836+0000 7fd11601ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:03.841 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:03.840+0000 7fd11601ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:03.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T12:47:03.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:47:03.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:47:03.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:47:03.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:47:03.970 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:47:03.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:03.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:47:03.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:03.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:47:04.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:47:04.237 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:04.236+0000 7fd11601ea40 -1 Falling back to public interface 2026-02-19T12:47:04.802 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:04.800+0000 7fd11601ea40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:47:05.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:47:05.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:05.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:47:05.132 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:47:05.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:05.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:47:05.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:47:06.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:47:06.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:06.291 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:47:06.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:47:06.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:06.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:47:06.449 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2295674225,v1:127.0.0.1:6811/2295674225] [v2:127.0.0.1:6812/2295674225,v1:127.0.0.1:6813/2295674225] exists,up 3b4b8f5e-82c1-403b-b67f-be60d495916c 2026-02-19T12:47:06.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:47:06.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:47:06.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:47:06.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:672: TEST_auto_repair_bluestore_failed_norecov: for id in $(seq 0 2) 2026-02-19T12:47:06.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:673: TEST_auto_repair_bluestore_failed_norecov: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:47:06.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:47:06.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:47:06.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T12:47:06.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:47:06.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:47:06.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:47:06.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:47:06.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:47:06.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:47:06.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:47:06.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:47:06.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:47:06.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:47:06.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:47:06.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:47:06.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:47:06.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:47:06.455 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:47:06.455 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:47:06.455 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:47:06.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:06.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:06.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:47:06.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:47:06.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:47:06.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:47:06.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:47:06.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:47:06.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:47:06.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:47:06.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:47:06.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:47:06.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:47:06.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:47:06.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:47:06.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b5d61f47-edfb-40d8-ae9f-cc54dffda7f1 2026-02-19T12:47:06.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 b5d61f47-edfb-40d8-ae9f-cc54dffda7f1' 2026-02-19T12:47:06.457 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 b5d61f47-edfb-40d8-ae9f-cc54dffda7f1 2026-02-19T12:47:06.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:47:06.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBKBpdp4UbMGxAAC0xsrXzLzEAoCGgD1mdufg== 2026-02-19T12:47:06.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBKBpdp4UbMGxAAC0xsrXzLzEAoCGgD1mdufg=="}' 2026-02-19T12:47:06.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b5d61f47-edfb-40d8-ae9f-cc54dffda7f1 -i td/osd-scrub-repair/2/new.json 2026-02-19T12:47:06.626 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:47:06.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T12:47:06.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQBKBpdp4UbMGxAAC0xsrXzLzEAoCGgD1mdufg== --osd-uuid b5d61f47-edfb-40d8-ae9f-cc54dffda7f1 2026-02-19T12:47:06.663 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:06.660+0000 7fcaaab02a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:06.665 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:06.664+0000 7fcaaab02a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:06.666 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:06.664+0000 7fcaaab02a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:06.666 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:06.664+0000 7fcaaab02a40 -1 bdev(0x558113e68c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T12:47:06.667 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:06.664+0000 7fcaaab02a40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T12:47:07.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T12:47:07.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:47:07.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T12:47:07.269 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T12:47:07.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:47:07.473 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T12:47:07.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T12:47:07.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:47:07.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:47:07.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:47:07.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:47:07.499 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:07.496+0000 7fee76c48a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:07.501 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:07.500+0000 7fee76c48a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:07.502 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:07.500+0000 7fee76c48a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:07.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T12:47:07.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:47:07.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:47:07.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:47:07.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:47:07.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:07.632 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:47:07.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:47:07.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:07.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:47:07.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:47:07.869 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:07.868+0000 7fee76c48a40 -1 Falling back to public interface 2026-02-19T12:47:08.446 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:08.444+0000 7fee76c48a40 -1 osd.2 0 log_to_monitors true 2026-02-19T12:47:08.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:47:08.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:08.794 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:47:08.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:47:08.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:08.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:47:08.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:47:09.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:47:09.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:09.951 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:47:09.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:47:09.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:09.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:47:10.114 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2074505501,v1:127.0.0.1:6819/2074505501] [v2:127.0.0.1:6820/2074505501,v1:127.0.0.1:6821/2074505501] exists,up b5d61f47-edfb-40d8-ae9f-cc54dffda7f1 2026-02-19T12:47:10.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:47:10.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:47:10.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:47:10.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:676: TEST_auto_repair_bluestore_failed_norecov: create_pool testpool 1 1 2026-02-19T12:47:10.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create testpool 1 1 2026-02-19T12:47:10.318 INFO:tasks.workunit.client.0.trial144.stderr:pool 'testpool' created 2026-02-19T12:47:10.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:47:11.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_auto_repair_bluestore_failed_norecov: ceph osd pool set testpool size 2 2026-02-19T12:47:11.535 INFO:tasks.workunit.client.0.trial144.stderr:set pool 1 size to 2 2026-02-19T12:47:11.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:678: TEST_auto_repair_bluestore_failed_norecov: wait_for_clean 2026-02-19T12:47:11.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:47:11.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:47:11.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:47:11.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:47:11.549 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:47:11.549 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:47:11.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:47:11.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:47:11.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:47:11.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:47:11.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:47:11.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:47:11.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:47:11.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:47:11.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:47:11.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:47:11.833 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:47:11.833 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:47:11.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:47:11.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:11.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:47:11.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T12:47:11.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T12:47:11.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T12:47:11.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:11.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:47:11.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-19T12:47:11.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-19T12:47:11.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705667' 2026-02-19T12:47:11.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:11.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:47:12.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574850 2026-02-19T12:47:12.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574850 2026-02-19T12:47:12.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705667 2-55834574850' 2026-02-19T12:47:12.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:12.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T12:47:12.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:12.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:47:12.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T12:47:12.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:12.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T12:47:12.062 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T12:47:12.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T12:47:12.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:12.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T12:47:12.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:47:13.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:47:13.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:13.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T12:47:13.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:47:14.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:47:14.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:14.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836483 2026-02-19T12:47:14.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:14.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-19T12:47:14.552 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:14.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:47:14.553 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705667 2026-02-19T12:47:14.553 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:14.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705667 2026-02-19T12:47:14.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-19T12:47:14.555 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705667 2026-02-19T12:47:14.556 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:47:14.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705667 -lt 38654705667 2026-02-19T12:47:14.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:14.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574850 2026-02-19T12:47:14.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:14.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:47:14.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574850 2026-02-19T12:47:14.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:14.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574850 2026-02-19T12:47:14.728 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574850 2026-02-19T12:47:14.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574850' 2026-02-19T12:47:14.729 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:47:14.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574850 -lt 55834574850 2026-02-19T12:47:14.893 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:47:14.893 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:14.893 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:15.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:47:15.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:47:15.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:47:15.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:47:15.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:47:15.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:47:15.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:47:15.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:47:15.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T12:47:15.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:47:15.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:15.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:15.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T12:47:15.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-19T12:47:15.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T12:47:15.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-19T12:47:15.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:47:15.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T12:47:15.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:47:15.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:47:15.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:47:15.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:47:15.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:47:15.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:47:15.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:47:15.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:47:15.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T12:47:15.689 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:47:15.690 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:15.690 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:15.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T12:47:15.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-19T12:47:15.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:47:15.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:47:15.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:47:15.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:47:15.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:47:15.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:47:15.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:47:16.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:47:16.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:47:16.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T12:47:16.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:47:16.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T12:47:16.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:47:16.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:47:16.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:47:16.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:47:16.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:47:16.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:47:16.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:47:16.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:47:16.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T12:47:16.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:47:16.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:16.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:16.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T12:47:16.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-19T12:47:16.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:47:16.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:47:16.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:47:16.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:47:16.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:47:16.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:47:16.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:47:16.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:47:16.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:47:16.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-19T12:47:16.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:47:16.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-19T12:47:17.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:47:17.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:47:17.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:47:17.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:47:17.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:47:17.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:47:17.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:47:17.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:47:17.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T12:47:17.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:47:17.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:17.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:17.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T12:47:17.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-19T12:47:17.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:47:17.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:47:17.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:47:17.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:47:17.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:47:17.571 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:47:17.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:47:17.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:47:17.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:47:17.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 13 )) 2026-02-19T12:47:17.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:47:17.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-19T12:47:18.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:47:18.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:47:18.571 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:47:18.571 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:47:18.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:47:18.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:47:18.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:47:18.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:47:18.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:47:18.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:47:18.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:18.732 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:18.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:47:18.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:47:18.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:47:18.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:681: TEST_auto_repair_bluestore_failed_norecov: local payload=ABCDEF 2026-02-19T12:47:18.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:682: TEST_auto_repair_bluestore_failed_norecov: echo ABCDEF 2026-02-19T12:47:18.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:683: TEST_auto_repair_bluestore_failed_norecov: seq 1 10 2026-02-19T12:47:18.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:683: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-19T12:47:18.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:685: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj1 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:47:18.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:683: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-19T12:47:18.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:685: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj2 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:47:18.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:683: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-19T12:47:18.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:685: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj3 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:47:18.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:683: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-19T12:47:18.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:685: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj4 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:47:19.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:683: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-19T12:47:19.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:685: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj5 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:47:19.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:683: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-19T12:47:19.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:685: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj6 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:47:19.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:683: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-19T12:47:19.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:685: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj7 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:47:19.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:683: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-19T12:47:19.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:685: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj8 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:47:19.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:683: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-19T12:47:19.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:685: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj9 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:47:19.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:683: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-19T12:47:19.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:685: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj10 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:47:19.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:691: TEST_auto_repair_bluestore_failed_norecov: get_not_primary testpool SOMETHING 2026-02-19T12:47:19.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=testpool 2026-02-19T12:47:19.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:47:19.163 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary testpool SOMETHING 2026-02-19T12:47:19.163 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:47:19.163 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:47:19.163 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:47:19.163 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:47:19.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:47:19.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:47:19.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:47:19.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:691: TEST_auto_repair_bluestore_failed_norecov: objectstore_tool td/osd-scrub-repair 0 obj1 remove 2026-02-19T12:47:19.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:47:19.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:47:19.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:47:19.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:47:19.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 obj1 remove 2026-02-19T12:47:19.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:47:19.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:47:19.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:47:19.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:47:19.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:47:19.480 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:47:19.480 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:47:19.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:47:19.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:47:19.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:47:19.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:47:19.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 obj1 remove 2026-02-19T12:47:19.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:47:19.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:47:19.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:47:19.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:47:19.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:47:19.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj1 remove 2026-02-19T12:47:19.960 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-19T12:47:20.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:47:20.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:47:20.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:47:20.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:47:20.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:47:20.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:47:20.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:47:20.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:47:20.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:47:20.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:47:20.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:47:20.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:47:20.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:47:20.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:47:20.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:47:20.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:47:20.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:47:20.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:47:20.069 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:47:20.069 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:47:20.069 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:47:20.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:20.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:20.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:47:20.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:47:20.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:47:20.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:47:20.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:47:20.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:47:20.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:47:20.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:47:20.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:47:20.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:47:20.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:47:20.070 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:47:20.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:47:20.072 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:47:20.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:47:20.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:47:20.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:47:20.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:47:20.087 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:20.083+0000 7f199dd4ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:20.088 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:20.087+0000 7f199dd4ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:20.089 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:20.087+0000 7f199dd4ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:20.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:47:20.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:47:20.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:47:20.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:47:20.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:47:20.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:20.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:47:20.225 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:47:20.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:20.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:47:20.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:47:20.449 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:20.447+0000 7f199dd4ca40 -1 Falling back to public interface 2026-02-19T12:47:21.060 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:21.055+0000 7f199dd4ca40 -1 osd.0 19 log_to_monitors true 2026-02-19T12:47:21.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:47:21.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:21.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:47:21.384 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:47:21.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:21.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:47:21.543 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 23 up_thru 0 down_at 20 last_clean_interval [5,19) [v2:127.0.0.1:6802/896985203,v1:127.0.0.1:6803/896985203] [v2:127.0.0.1:6804/896985203,v1:127.0.0.1:6805/896985203] exists,up 09bf220f-c5cd-4ce9-a942-3f1fbf430b5b 2026-02-19T12:47:21.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:47:21.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:47:21.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:47:21.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:47:21.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:47:21.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:47:21.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:47:21.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:47:21.545 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:47:21.545 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:47:21.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:47:21.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:47:21.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:47:21.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:47:21.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:47:21.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:47:21.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:47:21.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:47:21.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:47:21.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:47:21.805 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:47:21.805 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:47:21.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:47:21.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:21.806 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:47:21.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247810 2026-02-19T12:47:21.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247810 2026-02-19T12:47:21.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247810' 2026-02-19T12:47:21.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:21.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:47:21.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705670 2026-02-19T12:47:21.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705670 2026-02-19T12:47:21.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247810 1-38654705670' 2026-02-19T12:47:21.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:21.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:47:22.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574853 2026-02-19T12:47:22.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574853 2026-02-19T12:47:22.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247810 1-38654705670 2-55834574853' 2026-02-19T12:47:22.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:22.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-98784247810 2026-02-19T12:47:22.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:22.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:47:22.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-98784247810 2026-02-19T12:47:22.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:22.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=98784247810 2026-02-19T12:47:22.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 98784247810' 2026-02-19T12:47:22.026 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 98784247810 2026-02-19T12:47:22.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:22.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 98784247810 2026-02-19T12:47:22.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:47:23.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:47:23.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:23.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 98784247810 2026-02-19T12:47:23.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:47:24.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:47:24.357 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:24.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247810 -lt 98784247810 2026-02-19T12:47:24.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:24.517 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705670 2026-02-19T12:47:24.517 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:24.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:47:24.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705670 2026-02-19T12:47:24.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:24.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705670 2026-02-19T12:47:24.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-19T12:47:24.521 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 38654705670 2026-02-19T12:47:24.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:47:24.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705670 -lt 38654705670 2026-02-19T12:47:24.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:24.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574853 2026-02-19T12:47:24.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:24.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:47:24.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574853 2026-02-19T12:47:24.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:24.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574853 2026-02-19T12:47:24.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574853' 2026-02-19T12:47:24.688 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 55834574853 2026-02-19T12:47:24.689 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:47:24.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574853 -lt 55834574853 2026-02-19T12:47:24.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:47:24.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:24.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:25.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:47:25.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:47:25.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:47:25.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:47:25.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:47:25.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:47:25.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:47:25.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:47:25.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:47:25.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:47:25.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:25.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:25.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:47:25.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:47:25.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:47:25.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:692: TEST_auto_repair_bluestore_failed_norecov: get_primary testpool SOMETHING 2026-02-19T12:47:25.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:47:25.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:47:25.379 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:47:25.379 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:47:25.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:692: TEST_auto_repair_bluestore_failed_norecov: objectstore_tool td/osd-scrub-repair 1 obj1 rm-attr _ 2026-02-19T12:47:25.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:47:25.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:47:25.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:47:25.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:47:25.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 obj1 rm-attr _ 2026-02-19T12:47:25.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:47:25.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:47:25.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:47:25.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:47:25.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:47:25.536 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:47:25.537 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:47:25.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:47:25.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:47:25.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:47:25.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:47:25.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 obj1 rm-attr _ 2026-02-19T12:47:25.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:47:25.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:47:25.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:47:25.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:47:25.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:47:25.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj1 rm-attr _ 2026-02-19T12:47:26.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:47:26.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:47:26.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:47:26.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:47:26.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:47:26.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:47:26.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:47:26.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:47:26.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:47:26.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:47:26.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:47:26.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:47:26.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:47:26.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:47:26.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:47:26.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:47:26.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:47:26.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:47:26.176 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:47:26.176 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:47:26.176 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:47:26.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:26.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:26.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:47:26.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:47:26.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:47:26.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:47:26.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:47:26.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:47:26.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:47:26.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:47:26.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:47:26.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:47:26.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:47:26.179 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:47:26.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:47:26.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:47:26.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:47:26.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:47:26.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:47:26.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:47:26.195 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:26.191+0000 7f5f415a2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:26.196 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:26.195+0000 7f5f415a2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:26.197 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:26.195+0000 7f5f415a2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:26.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:47:26.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:47:26.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:47:26.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:47:26.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:47:26.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:26.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:47:26.333 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:47:26.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:26.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:47:26.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:47:26.605 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:26.603+0000 7f5f415a2a40 -1 Falling back to public interface 2026-02-19T12:47:27.190 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:27.187+0000 7f5f415a2a40 -1 osd.1 25 log_to_monitors true 2026-02-19T12:47:27.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:47:27.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:27.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:47:27.496 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:47:27.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:27.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:47:27.651 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 29 up_thru 29 down_at 26 last_clean_interval [9,25) [v2:127.0.0.1:6810/1567515797,v1:127.0.0.1:6811/1567515797] [v2:127.0.0.1:6812/1567515797,v1:127.0.0.1:6813/1567515797] exists,up 3b4b8f5e-82c1-403b-b67f-be60d495916c 2026-02-19T12:47:27.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:47:27.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:47:27.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:47:27.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:47:27.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:47:27.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:47:27.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:47:27.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:47:27.652 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:47:27.652 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:47:27.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:47:27.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:47:27.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:47:27.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:47:27.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:47:27.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:47:27.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:47:27.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:47:27.766 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:47:27.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:47:27.928 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:47:27.928 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:47:27.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:47:27.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:27.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:47:28.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247812 2026-02-19T12:47:28.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247812 2026-02-19T12:47:28.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247812' 2026-02-19T12:47:28.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:28.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:47:28.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051586 2026-02-19T12:47:28.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051586 2026-02-19T12:47:28.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247812 1-124554051586' 2026-02-19T12:47:28.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:28.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:47:28.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-19T12:47:28.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-19T12:47:28.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247812 1-124554051586 2-55834574855' 2026-02-19T12:47:28.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:28.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-98784247812 2026-02-19T12:47:28.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:28.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:47:28.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-98784247812 2026-02-19T12:47:28.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:28.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=98784247812 2026-02-19T12:47:28.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 98784247812' 2026-02-19T12:47:28.153 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 98784247812 2026-02-19T12:47:28.153 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:28.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247811 -lt 98784247812 2026-02-19T12:47:28.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:47:29.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:47:29.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:29.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247811 -lt 98784247812 2026-02-19T12:47:29.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:47:30.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:47:30.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:30.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247812 -lt 98784247812 2026-02-19T12:47:30.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:30.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-124554051586 2026-02-19T12:47:30.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:30.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:47:30.635 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-124554051586 2026-02-19T12:47:30.636 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:30.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051586 2026-02-19T12:47:30.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 124554051586' 2026-02-19T12:47:30.637 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 124554051586 2026-02-19T12:47:30.637 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:47:30.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051586 -lt 124554051586 2026-02-19T12:47:30.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:30.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574855 2026-02-19T12:47:30.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:30.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:47:30.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574855 2026-02-19T12:47:30.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:30.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574855 2026-02-19T12:47:30.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574855' 2026-02-19T12:47:30.806 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 55834574855 2026-02-19T12:47:30.806 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:47:30.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-19T12:47:30.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:47:30.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:30.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:31.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:47:31.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:47:31.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:47:31.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:47:31.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:47:31.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:47:31.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:47:31.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:47:31.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:47:31.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:47:31.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:31.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:31.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:47:31.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:47:31.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:47:31.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_auto_repair_bluestore_failed_norecov: get_not_primary testpool SOMETHING 2026-02-19T12:47:31.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=testpool 2026-02-19T12:47:31.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:47:31.522 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary testpool SOMETHING 2026-02-19T12:47:31.522 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:47:31.522 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:47:31.522 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:47:31.522 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:47:31.678 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:47:31.679 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:47:31.679 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:47:31.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_auto_repair_bluestore_failed_norecov: objectstore_tool td/osd-scrub-repair 0 obj2 remove 2026-02-19T12:47:31.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:47:31.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:47:31.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:47:31.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:47:31.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 obj2 remove 2026-02-19T12:47:31.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:47:31.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:47:31.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:47:31.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:47:31.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:47:31.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:47:31.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:47:31.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:47:31.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:47:31.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:47:32.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:47:32.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 obj2 remove 2026-02-19T12:47:32.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:47:32.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:47:32.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:47:32.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:47:32.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:47:32.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj2 remove 2026-02-19T12:47:32.311 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:104778fc:::obj2:head# 2026-02-19T12:47:32.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:47:32.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:47:32.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:47:32.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:47:32.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:47:32.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:47:32.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:47:32.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:47:32.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:47:32.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:47:32.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:47:32.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:47:32.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:47:32.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:47:32.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:47:32.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:47:32.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:47:32.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:47:32.408 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:47:32.408 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:47:32.409 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:47:32.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:32.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:32.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:47:32.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:47:32.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:47:32.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:47:32.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:47:32.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:47:32.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:47:32.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:47:32.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:47:32.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:47:32.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:47:32.411 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:47:32.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:47:32.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:47:32.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:47:32.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:47:32.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:47:32.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:47:32.426 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:32.423+0000 7fd5a9974a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:32.427 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:32.423+0000 7fd5a9974a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:32.428 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:32.427+0000 7fd5a9974a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:32.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:47:32.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:47:32.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:47:32.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:47:32.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:47:32.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:32.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:47:32.564 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:47:32.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:32.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:47:32.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:47:32.829 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:32.827+0000 7fd5a9974a40 -1 Falling back to public interface 2026-02-19T12:47:33.446 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:33.443+0000 7fd5a9974a40 -1 osd.0 31 log_to_monitors true 2026-02-19T12:47:33.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:47:33.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:33.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:47:33.725 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:47:33.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:33.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:47:33.881 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 35 up_thru 26 down_at 32 last_clean_interval [23,31) [v2:127.0.0.1:6802/3390499279,v1:127.0.0.1:6803/3390499279] [v2:127.0.0.1:6804/3390499279,v1:127.0.0.1:6805/3390499279] exists,up 09bf220f-c5cd-4ce9-a942-3f1fbf430b5b 2026-02-19T12:47:33.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:47:33.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:47:33.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:47:33.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:47:33.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:47:33.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:47:33.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:47:33.882 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:47:33.883 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:47:33.883 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:47:33.883 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:47:33.884 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:47:33.884 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:47:33.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:47:33.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:47:33.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:47:33.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:47:33.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:47:33.995 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:47:34.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:47:34.158 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:47:34.158 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:47:34.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:47:34.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:34.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:47:34.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855362 2026-02-19T12:47:34.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855362 2026-02-19T12:47:34.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855362' 2026-02-19T12:47:34.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:34.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:47:34.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051588 2026-02-19T12:47:34.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051588 2026-02-19T12:47:34.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855362 1-124554051588' 2026-02-19T12:47:34.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:34.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:47:34.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-19T12:47:34.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-19T12:47:34.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855362 1-124554051588 2-55834574857' 2026-02-19T12:47:34.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:34.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-150323855362 2026-02-19T12:47:34.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:34.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:47:34.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-150323855362 2026-02-19T12:47:34.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:34.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855362 2026-02-19T12:47:34.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 150323855362' 2026-02-19T12:47:34.381 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 150323855362 2026-02-19T12:47:34.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:34.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247813 -lt 150323855362 2026-02-19T12:47:34.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:47:35.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:47:35.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:35.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855362 -lt 150323855362 2026-02-19T12:47:35.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:35.705 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-124554051588 2026-02-19T12:47:35.705 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:35.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:47:35.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-124554051588 2026-02-19T12:47:35.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:35.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051588 2026-02-19T12:47:35.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 124554051588' 2026-02-19T12:47:35.709 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 124554051588 2026-02-19T12:47:35.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:47:35.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051588 -lt 124554051588 2026-02-19T12:47:35.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:35.877 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574857 2026-02-19T12:47:35.877 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:35.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:47:35.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574857 2026-02-19T12:47:35.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:35.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574857 2026-02-19T12:47:35.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-19T12:47:35.881 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 55834574857 2026-02-19T12:47:35.881 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:47:36.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574858 -lt 55834574857 2026-02-19T12:47:36.039 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:47:36.039 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:36.039 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:36.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:47:36.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:47:36.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:47:36.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:47:36.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:47:36.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:47:36.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:47:36.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:47:36.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:47:36.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:47:36.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:36.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:36.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:47:36.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:47:36.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:47:36.564 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:695: TEST_auto_repair_bluestore_failed_norecov: get_primary testpool SOMETHING 2026-02-19T12:47:36.564 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:47:36.564 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:47:36.564 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:47:36.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:47:36.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:695: TEST_auto_repair_bluestore_failed_norecov: objectstore_tool td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-19T12:47:36.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:47:36.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:47:36.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:47:36.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:47:36.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-19T12:47:36.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:47:36.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:47:36.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:47:36.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:47:36.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:47:36.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:47:36.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:47:36.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:47:36.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:47:36.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:47:37.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:47:37.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-19T12:47:37.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:47:37.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:47:37.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:47:37.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:47:37.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:47:37.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj2 rm-attr _ 2026-02-19T12:47:37.191 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:ff7b1f36:::obj1:head#, (61) No data available 2026-02-19T12:47:37.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:47:37.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:47:37.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:47:37.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:47:37.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:47:37.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:47:37.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:47:37.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:47:37.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:47:37.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:47:37.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:47:37.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:47:37.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:47:37.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:47:37.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:47:37.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:47:37.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:47:37.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:47:37.280 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:47:37.280 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:47:37.280 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:47:37.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:37.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:37.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:47:37.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:47:37.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:47:37.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:47:37.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:47:37.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:47:37.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:47:37.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:47:37.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:47:37.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:47:37.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:47:37.282 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:47:37.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:47:37.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:47:37.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:47:37.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:47:37.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:47:37.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:47:37.300 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:37.299+0000 7f7d404c4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:37.301 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:37.299+0000 7f7d404c4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:37.303 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:37.299+0000 7f7d404c4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:37.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:47:37.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:47:37.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:47:37.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:47:37.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:47:37.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:37.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:47:37.437 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:47:37.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:37.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:47:37.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:47:37.677 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:37.675+0000 7f7d404c4a40 -1 Falling back to public interface 2026-02-19T12:47:38.266 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:38.263+0000 7f7d404c4a40 -1 osd.1 36 log_to_monitors true 2026-02-19T12:47:38.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:47:38.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:38.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:47:38.601 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:47:38.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:38.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:47:38.759 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 40 up_thru 40 down_at 37 last_clean_interval [29,36) [v2:127.0.0.1:6810/3022849137,v1:127.0.0.1:6811/3022849137] [v2:127.0.0.1:6812/3022849137,v1:127.0.0.1:6813/3022849137] exists,up 3b4b8f5e-82c1-403b-b67f-be60d495916c 2026-02-19T12:47:38.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:47:38.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:47:38.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:47:38.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:47:38.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:47:38.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:47:38.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:47:38.760 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:47:38.761 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:47:38.761 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:47:38.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:47:38.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:47:38.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:47:38.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:47:38.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:47:38.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:47:38.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:47:38.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:47:38.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:47:39.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:47:39.032 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:47:39.032 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:47:39.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:47:39.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:39.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:47:39.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855364 2026-02-19T12:47:39.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855364 2026-02-19T12:47:39.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855364' 2026-02-19T12:47:39.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:39.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:47:39.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=171798691842 2026-02-19T12:47:39.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 171798691842 2026-02-19T12:47:39.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855364 1-171798691842' 2026-02-19T12:47:39.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:39.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:47:39.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574859 2026-02-19T12:47:39.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574859 2026-02-19T12:47:39.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855364 1-171798691842 2-55834574859' 2026-02-19T12:47:39.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:39.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-150323855364 2026-02-19T12:47:39.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:39.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:47:39.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-150323855364 2026-02-19T12:47:39.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:39.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855364 2026-02-19T12:47:39.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 150323855364' 2026-02-19T12:47:39.255 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 150323855364 2026-02-19T12:47:39.255 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:39.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855362 -lt 150323855364 2026-02-19T12:47:39.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:47:40.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:47:40.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:40.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855364 -lt 150323855364 2026-02-19T12:47:40.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:40.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-171798691842 2026-02-19T12:47:40.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:40.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:47:40.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-171798691842 2026-02-19T12:47:40.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:40.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=171798691842 2026-02-19T12:47:40.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 171798691842' 2026-02-19T12:47:40.592 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 171798691842 2026-02-19T12:47:40.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:47:40.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 171798691842 -lt 171798691842 2026-02-19T12:47:40.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:40.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574859 2026-02-19T12:47:40.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:40.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:47:40.764 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574859 2026-02-19T12:47:40.764 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:40.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574859 2026-02-19T12:47:40.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574859' 2026-02-19T12:47:40.765 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 55834574859 2026-02-19T12:47:40.766 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:47:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574859 -lt 55834574859 2026-02-19T12:47:40.937 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:47:40.937 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:40.937 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:41.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:47:41.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:47:41.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:47:41.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:47:41.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:47:41.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:47:41.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:47:41.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:47:41.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:47:41.290 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:47:41.290 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:41.290 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:41.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:47:41.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:47:41.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:47:41.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:696: TEST_auto_repair_bluestore_failed_norecov: ceph tell 'osd.*' config set osd_scrub_auto_repair true 2026-02-19T12:47:41.547 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T12:47:41.547 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_auto_repair = '' (not observed, change may require restart) osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-19T12:47:41.547 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:47:41.554 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T12:47:41.554 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_auto_repair = '' (not observed, change may require restart) osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-19T12:47:41.554 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:47:41.561 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T12:47:41.561 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_auto_repair = '' (not observed, change may require restart) " 2026-02-19T12:47:41.561 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:47:41.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:698: TEST_auto_repair_bluestore_failed_norecov: get_pg testpool obj1 2026-02-19T12:47:41.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=testpool 2026-02-19T12:47:41.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=obj1 2026-02-19T12:47:41.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map testpool obj1 2026-02-19T12:47:41.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:47:41.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:698: TEST_auto_repair_bluestore_failed_norecov: local pgid=1.0 2026-02-19T12:47:41.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:699: TEST_auto_repair_bluestore_failed_norecov: get_primary testpool obj1 2026-02-19T12:47:41.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:47:41.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T12:47:41.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool obj1 2026-02-19T12:47:41.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:47:41.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:699: TEST_auto_repair_bluestore_failed_norecov: local primary=1 2026-02-19T12:47:41.885 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:700: TEST_auto_repair_bluestore_failed_norecov: get_last_scrub_stamp 1.0 2026-02-19T12:47:41.885 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:47:41.885 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:47:41.885 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:47:41.885 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:47:42.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:700: TEST_auto_repair_bluestore_failed_norecov: local last_scrub_stamp=2026-02-19T12:47:10.317220+0000 2026-02-19T12:47:42.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:701: TEST_auto_repair_bluestore_failed_norecov: ceph tell 1.0 schedule-deep-scrub 2026-02-19T12:47:42.105 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T12:47:42.105 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T12:47:42.105 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T12:47:42.105 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-08T07:59:32.105225+0000" 2026-02-19T12:47:42.105 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:47:42.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:704: TEST_auto_repair_bluestore_failed_norecov: wait_for_scrub 1.0 2026-02-19T12:47:10.317220+0000 2026-02-19T12:47:42.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T12:47:42.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:47:10.317220+0000 2026-02-19T12:47:42.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:47:42.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:47:42.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:47:42.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:47:42.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:47:42.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:47:42.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:47:42.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:47:42.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:47:10.317220+0000 '>' 2026-02-19T12:47:10.317220+0000 2026-02-19T12:47:42.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:47:43.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:47:43.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:47:43.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:47:43.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:47:43.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:47:43.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:47:43.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:47:43.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:47:10.317220+0000 '>' 2026-02-19T12:47:10.317220+0000 2026-02-19T12:47:43.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:47:44.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:47:44.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:47:44.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:47:44.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:47:44.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:47:44.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:47:44.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:47:44.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:47:10.317220+0000 '>' 2026-02-19T12:47:10.317220+0000 2026-02-19T12:47:44.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:47:45.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:47:45.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:47:45.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:47:45.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:47:45.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:47:45.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:47:45.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:47:45.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:47:10.317220+0000 '>' 2026-02-19T12:47:10.317220+0000 2026-02-19T12:47:45.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:47:46.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:47:46.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:47:46.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:47:46.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:47:46.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:47:46.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:47:46.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:47:46.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:47:10.317220+0000 '>' 2026-02-19T12:47:10.317220+0000 2026-02-19T12:47:46.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:47:47.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:47:47.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:47:47.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:47:47.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:47:47.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:47:47.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:47:47.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:47:48.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:47:10.317220+0000 '>' 2026-02-19T12:47:10.317220+0000 2026-02-19T12:47:48.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:47:49.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:47:49.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:47:49.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:47:49.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:47:49.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:47:49.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:47:49.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:47:49.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:47:10.317220+0000 '>' 2026-02-19T12:47:10.317220+0000 2026-02-19T12:47:49.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:47:50.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:47:50.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:47:50.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:47:50.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:47:50.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:47:50.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:47:50.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:47:50.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:47:47.309597+0000 '>' 2026-02-19T12:47:10.317220+0000 2026-02-19T12:47:50.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:47:50.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:705: TEST_auto_repair_bluestore_failed_norecov: wait_for_clean 2026-02-19T12:47:50.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:47:50.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:47:50.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:47:50.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:47:50.394 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:47:50.394 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:47:50.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:47:50.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:47:50.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:47:50.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:47:50.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:47:50.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:47:50.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:47:50.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:47:50.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:47:50.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:47:50.659 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:47:50.659 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:47:50.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:47:50.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:50.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:47:50.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855367 2026-02-19T12:47:50.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855367 2026-02-19T12:47:50.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855367' 2026-02-19T12:47:50.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:50.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:47:50.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=171798691845 2026-02-19T12:47:50.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 171798691845 2026-02-19T12:47:50.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855367 1-171798691845' 2026-02-19T12:47:50.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:50.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:47:50.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574863 2026-02-19T12:47:50.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574863 2026-02-19T12:47:50.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855367 1-171798691845 2-55834574863' 2026-02-19T12:47:50.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:50.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-150323855367 2026-02-19T12:47:50.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:50.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:47:50.877 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-150323855367 2026-02-19T12:47:50.877 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:50.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855367 2026-02-19T12:47:50.879 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 150323855367 2026-02-19T12:47:50.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 150323855367' 2026-02-19T12:47:50.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:51.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855366 -lt 150323855367 2026-02-19T12:47:51.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:47:52.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:47:52.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:52.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855367 -lt 150323855367 2026-02-19T12:47:52.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:52.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-171798691845 2026-02-19T12:47:52.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:52.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:47:52.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-171798691845 2026-02-19T12:47:52.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:52.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=171798691845 2026-02-19T12:47:52.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 171798691845' 2026-02-19T12:47:52.203 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 171798691845 2026-02-19T12:47:52.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:47:52.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 171798691845 -lt 171798691845 2026-02-19T12:47:52.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:52.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574863 2026-02-19T12:47:52.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:52.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:47:52.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574863 2026-02-19T12:47:52.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:52.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574863 2026-02-19T12:47:52.376 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574863 2026-02-19T12:47:52.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574863' 2026-02-19T12:47:52.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:47:52.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574863 -lt 55834574863 2026-02-19T12:47:52.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:47:52.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:52.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:52.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:47:52.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:47:52.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:47:52.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:47:52.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:47:52.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:47:52.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:47:52.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:47:52.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:47:52.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:47:52.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:47:52.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:47:53.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:47:53.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:47:53.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:47:53.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:706: TEST_auto_repair_bluestore_failed_norecov: flush_pg_stats 2026-02-19T12:47:53.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:47:53.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:47:53.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:47:53.241 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:47:53.241 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:47:53.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:47:53.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:53.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:47:53.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855368 2026-02-19T12:47:53.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855368 2026-02-19T12:47:53.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855368' 2026-02-19T12:47:53.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:53.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:47:53.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=171798691847 2026-02-19T12:47:53.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 171798691847 2026-02-19T12:47:53.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855368 1-171798691847' 2026-02-19T12:47:53.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:47:53.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:47:53.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574864 2026-02-19T12:47:53.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574864 2026-02-19T12:47:53.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855368 1-171798691847 2-55834574864' 2026-02-19T12:47:53.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:53.463 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-150323855368 2026-02-19T12:47:53.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:53.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:47:53.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-150323855368 2026-02-19T12:47:53.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:53.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855368 2026-02-19T12:47:53.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 150323855368' 2026-02-19T12:47:53.467 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 150323855368 2026-02-19T12:47:53.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:47:53.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855368 -lt 150323855368 2026-02-19T12:47:53.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:53.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-171798691847 2026-02-19T12:47:53.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:53.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:47:53.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-171798691847 2026-02-19T12:47:53.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:53.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=171798691847 2026-02-19T12:47:53.628 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 171798691847 2026-02-19T12:47:53.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 171798691847' 2026-02-19T12:47:53.628 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:47:53.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 171798691847 -lt 171798691847 2026-02-19T12:47:53.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:47:53.788 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574864 2026-02-19T12:47:53.788 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:47:53.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:47:53.791 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574864 2026-02-19T12:47:53.791 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:47:53.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574864 2026-02-19T12:47:53.792 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574864 2026-02-19T12:47:53.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574864' 2026-02-19T12:47:53.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:47:53.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574864 -lt 55834574864 2026-02-19T12:47:53.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:707: TEST_auto_repair_bluestore_failed_norecov: grep -q 'scrub_finish.*present with no repair possible' td/osd-scrub-repair/osd.1.log 2026-02-19T12:47:53.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:708: TEST_auto_repair_bluestore_failed_norecov: ceph pg dump pgs 2026-02-19T12:47:54.109 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:47:54.109 INFO:tasks.workunit.client.0.trial144.stdout:1.0 10 0 0 0 0 56 0 0 10 0 10 active+clean+inconsistent+failed_repair 2026-02-19T12:47:47.309668+0000 19'10 41:104 [1,0] 1 [1,0] 1 19'10 2026-02-19T12:47:47.309597+0000 19'10 2026-02-19T12:47:47.309597+0000 0 1 periodic scrub scheduled @ 2026-02-20T12:47:47.309597+0000 10 0 2026-02-19T12:47:54.110 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:47:54.110 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:47:54.110 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:47:54.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:709: TEST_auto_repair_bluestore_failed_norecov: ceph pg dump pgs 2026-02-19T12:47:54.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:709: TEST_auto_repair_bluestore_failed_norecov: grep -q '^1.0.*+failed_repair' 2026-02-19T12:47:54.276 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:47:54.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T12:47:54.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:47:54.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:47:54.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:47:54.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:47:54.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:47:54.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:47:54.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:47:54.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:47:54.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:47:54.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:47:54.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:47:54.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:47:54.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:47:54.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:47:54.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:47:54.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:47:54.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:47:54.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:47:54.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:47:54.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:47:54.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:47:54.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:47:54.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:47:54.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:47:54.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:47:54.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:47:54.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:47:54.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:47:54.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T12:47:54.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T12:47:54.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T12:47:54.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T12:47:54.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:47:54.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:47:54.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:47:54.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:47:54.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:47:54.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:47:54.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:47:54.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:47:54.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:47:54.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:47:54.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:47:54.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:47:54.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:47:54.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:47:54.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:47:54.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:47:54.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:47:54.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:47:54.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:47:54.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:47:54.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:47:54.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:47:54.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:47:54.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:47:54.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:47:54.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:47:54.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:47:54.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:47:54.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T12:47:54.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:47:54.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:47:54.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:47:54.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T12:47:54.440 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:47:54.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T12:47:54.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:47:54.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T12:47:54.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_bluestore_scrub td/osd-scrub-repair 2026-02-19T12:47:54.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:542: TEST_auto_repair_bluestore_scrub: local dir=td/osd-scrub-repair 2026-02-19T12:47:54.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:543: TEST_auto_repair_bluestore_scrub: local poolname=testpool 2026-02-19T12:47:54.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:546: TEST_auto_repair_bluestore_scrub: run_mon td/osd-scrub-repair a 2026-02-19T12:47:54.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T12:47:54.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T12:47:54.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T12:47:54.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T12:47:54.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T12:47:54.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T12:47:54.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T12:47:54.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:47:54.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:47:54.465 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:47:54.465 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:47:54.465 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:47:54.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:54.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T12:47:54.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T12:47:54.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T12:47:54.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:47:54.496 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:47:54.496 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T12:47:54.496 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T12:47:54.496 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:47:54.496 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:47:54.496 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:47:54.497 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:47:54.497 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:47:54.497 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:47:54.497 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:47:54.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:47:54.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T12:47:54.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T12:47:54.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:47:54.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:47:54.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T12:47:54.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T12:47:54.549 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:47:54.549 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:47:54.549 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:47:54.549 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:47:54.549 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:47:54.549 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:47:54.550 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:47:54.550 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:47:54.550 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T12:47:54.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:547: TEST_auto_repair_bluestore_scrub: run_mgr td/osd-scrub-repair x 2026-02-19T12:47:54.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T12:47:54.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T12:47:54.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T12:47:54.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T12:47:54.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T12:47:54.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T12:47:54.701 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T12:47:54.701 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:47:54.702 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:47:54.702 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:47:54.702 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:47:54.702 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:47:54.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:54.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:47:54.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:47:54.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:551: TEST_auto_repair_bluestore_scrub: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-19T12:47:54.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:552: TEST_auto_repair_bluestore_scrub: seq 0 2 2026-02-19T12:47:54.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:552: TEST_auto_repair_bluestore_scrub: for id in $(seq 0 2) 2026-02-19T12:47:54.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:553: TEST_auto_repair_bluestore_scrub: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-19T12:47:54.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:47:54.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:47:54.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T12:47:54.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:47:54.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:47:54.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:47:54.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:47:54.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:47:54.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:47:54.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:47:54.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:47:54.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:47:54.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:47:54.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:47:54.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:47:54.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:47:54.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:47:54.727 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:47:54.727 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:47:54.727 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:47:54.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:54.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:54.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:47:54.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:47:54.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:47:54.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:47:54.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:47:54.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:47:54.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:47:54.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:47:54.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:47:54.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:47:54.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-19T12:47:54.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:47:54.729 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:47:54.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=42b4dfbd-280e-4a0a-9f9d-28b259a62af8 2026-02-19T12:47:54.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 42b4dfbd-280e-4a0a-9f9d-28b259a62af8' 2026-02-19T12:47:54.729 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 42b4dfbd-280e-4a0a-9f9d-28b259a62af8 2026-02-19T12:47:54.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:47:54.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB6Bpdp9s7jKxAA66K7e1soXPV0EPdO/u94bw== 2026-02-19T12:47:54.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB6Bpdp9s7jKxAA66K7e1soXPV0EPdO/u94bw=="}' 2026-02-19T12:47:54.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 42b4dfbd-280e-4a0a-9f9d-28b259a62af8 -i td/osd-scrub-repair/0/new.json 2026-02-19T12:47:54.826 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:47:54.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T12:47:54.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 --mkfs --key AQB6Bpdp9s7jKxAA66K7e1soXPV0EPdO/u94bw== --osd-uuid 42b4dfbd-280e-4a0a-9f9d-28b259a62af8 2026-02-19T12:47:54.852 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:54.851+0000 7f8bb520ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:54.854 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:54.851+0000 7f8bb520ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:54.854 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:54.851+0000 7f8bb520ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:54.855 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:54.851+0000 7f8bb520ea40 -1 bdev(0x564284032c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:47:54.855 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:54.851+0000 7f8bb520ea40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T12:47:55.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T12:47:55.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:47:55.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T12:47:55.608 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:47:55.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:47:55.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T12:47:55.809 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:47:55.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-19T12:47:55.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:47:55.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:47:55.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:47:55.835 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:55.831+0000 7f72b2c07a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:55.837 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:55.835+0000 7f72b2c07a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:55.838 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:55.835+0000 7f72b2c07a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:55.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T12:47:55.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:47:55.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:47:55.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:47:55.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:47:55.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:55.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:47:55.979 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:47:55.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:55.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:47:56.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:47:56.257 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:56.255+0000 7f72b2c07a40 -1 Falling back to public interface 2026-02-19T12:47:56.877 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:56.875+0000 7f72b2c07a40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:47:57.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:47:57.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:57.136 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:47:57.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:47:57.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:57.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:47:57.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:47:58.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:47:58.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:58.295 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:47:58.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:47:58.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:58.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:47:58.450 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/585196948,v1:127.0.0.1:6803/585196948] [v2:127.0.0.1:6804/585196948,v1:127.0.0.1:6805/585196948] exists,up 42b4dfbd-280e-4a0a-9f9d-28b259a62af8 2026-02-19T12:47:58.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:47:58.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:47:58.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:47:58.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:552: TEST_auto_repair_bluestore_scrub: for id in $(seq 0 2) 2026-02-19T12:47:58.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:553: TEST_auto_repair_bluestore_scrub: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-19T12:47:58.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:47:58.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:47:58.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T12:47:58.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:47:58.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:47:58.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:47:58.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:47:58.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:47:58.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:47:58.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:47:58.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:47:58.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:47:58.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:47:58.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:47:58.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:47:58.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:47:58.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:47:58.462 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:47:58.462 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:47:58.462 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:47:58.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:58.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:47:58.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:47:58.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:47:58.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:47:58.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:47:58.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:47:58.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:47:58.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:47:58.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:47:58.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:47:58.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:47:58.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-19T12:47:58.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:47:58.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:47:58.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9f1d1f57-685a-4d30-b092-b9792b4a0b7a 2026-02-19T12:47:58.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 9f1d1f57-685a-4d30-b092-b9792b4a0b7a' 2026-02-19T12:47:58.466 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 9f1d1f57-685a-4d30-b092-b9792b4a0b7a 2026-02-19T12:47:58.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:47:58.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB+BpdpcYzPGxAAj5US21BE3JA0dXVF+7144w== 2026-02-19T12:47:58.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB+BpdpcYzPGxAAj5US21BE3JA0dXVF+7144w=="}' 2026-02-19T12:47:58.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9f1d1f57-685a-4d30-b092-b9792b4a0b7a -i td/osd-scrub-repair/1/new.json 2026-02-19T12:47:58.616 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:47:58.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T12:47:58.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 --mkfs --key AQB+BpdpcYzPGxAAj5US21BE3JA0dXVF+7144w== --osd-uuid 9f1d1f57-685a-4d30-b092-b9792b4a0b7a 2026-02-19T12:47:58.643 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:58.643+0000 7f7be38d4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:58.645 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:58.643+0000 7f7be38d4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:58.646 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:58.643+0000 7f7be38d4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:58.646 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:58.643+0000 7f7be38d4a40 -1 bdev(0x5643e98e8c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:47:58.646 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:58.643+0000 7f7be38d4a40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T12:47:59.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T12:47:59.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:47:59.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T12:47:59.264 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:47:59.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:47:59.460 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:47:59.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T12:47:59.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-19T12:47:59.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:47:59.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:47:59.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:47:59.486 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:59.483+0000 7f09e9df2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:59.487 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:59.487+0000 7f09e9df2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:59.489 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:59.487+0000 7f09e9df2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:47:59.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T12:47:59.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:47:59.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:47:59.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:47:59.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:47:59.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:47:59.624 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:47:59.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:47:59.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:47:59.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:47:59.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:47:59.833 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:47:59.831+0000 7f09e9df2a40 -1 Falling back to public interface 2026-02-19T12:48:00.436 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:00.435+0000 7f09e9df2a40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:48:00.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:48:00.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:48:00.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:48:00.786 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:48:00.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:48:00.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:48:00.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:48:01.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:48:01.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:48:01.947 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:48:01.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:48:01.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:48:01.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:48:02.104 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3935720174,v1:127.0.0.1:6811/3935720174] [v2:127.0.0.1:6812/3935720174,v1:127.0.0.1:6813/3935720174] exists,up 9f1d1f57-685a-4d30-b092-b9792b4a0b7a 2026-02-19T12:48:02.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:48:02.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:48:02.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:48:02.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:552: TEST_auto_repair_bluestore_scrub: for id in $(seq 0 2) 2026-02-19T12:48:02.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:553: TEST_auto_repair_bluestore_scrub: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-19T12:48:02.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:48:02.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:48:02.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T12:48:02.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:48:02.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:48:02.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:48:02.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:48:02.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:48:02.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:48:02.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:48:02.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:48:02.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:48:02.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:48:02.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:48:02.109 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:48:02.109 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:48:02.109 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:48:02.109 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:48:02.110 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:48:02.110 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:48:02.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:48:02.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:48:02.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:48:02.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:48:02.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:48:02.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:48:02.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:48:02.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:48:02.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:48:02.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:48:02.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:48:02.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:48:02.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-19T12:48:02.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:48:02.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:48:02.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c9c6c496-b495-4ada-b26d-43836e7fc512 2026-02-19T12:48:02.112 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 c9c6c496-b495-4ada-b26d-43836e7fc512 2026-02-19T12:48:02.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 c9c6c496-b495-4ada-b26d-43836e7fc512' 2026-02-19T12:48:02.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:48:02.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCCBpdpx45GBxAA/5kGfRj9PQxvo2TNlhGUTg== 2026-02-19T12:48:02.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCCBpdpx45GBxAA/5kGfRj9PQxvo2TNlhGUTg=="}' 2026-02-19T12:48:02.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c9c6c496-b495-4ada-b26d-43836e7fc512 -i td/osd-scrub-repair/2/new.json 2026-02-19T12:48:02.280 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:48:02.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T12:48:02.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 --mkfs --key AQCCBpdpx45GBxAA/5kGfRj9PQxvo2TNlhGUTg== --osd-uuid c9c6c496-b495-4ada-b26d-43836e7fc512 2026-02-19T12:48:02.308 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:02.307+0000 7fedcfe61a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:02.309 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:02.307+0000 7fedcfe61a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:02.310 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:02.307+0000 7fedcfe61a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:02.311 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:02.307+0000 7fedcfe61a40 -1 bdev(0x55b9ffd9ec00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T12:48:02.311 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:02.307+0000 7fedcfe61a40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T12:48:02.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T12:48:02.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:48:02.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T12:48:02.920 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T12:48:02.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:48:03.122 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T12:48:03.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T12:48:03.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-19T12:48:03.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:48:03.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:48:03.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:48:03.139 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:03.139+0000 7f53e2b99a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:03.141 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:03.139+0000 7f53e2b99a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:03.142 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:03.139+0000 7f53e2b99a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:03.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T12:48:03.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:48:03.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:48:03.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:48:03.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:48:03.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:48:03.286 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:48:03.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:48:03.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:48:03.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:48:03.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:48:03.553 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:03.551+0000 7f53e2b99a40 -1 Falling back to public interface 2026-02-19T12:48:04.135 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:04.131+0000 7f53e2b99a40 -1 osd.2 0 log_to_monitors true 2026-02-19T12:48:04.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:48:04.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:48:04.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:48:04.443 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:48:04.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:48:04.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:48:04.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:48:05.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:48:05.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:48:05.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:48:05.604 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:48:05.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:48:05.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:48:05.760 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4164233877,v1:127.0.0.1:6819/4164233877] [v2:127.0.0.1:6820/4164233877,v1:127.0.0.1:6821/4164233877] exists,up c9c6c496-b495-4ada-b26d-43836e7fc512 2026-02-19T12:48:05.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:48:05.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:48:05.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:48:05.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:556: TEST_auto_repair_bluestore_scrub: create_pool testpool 1 1 2026-02-19T12:48:05.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create testpool 1 1 2026-02-19T12:48:05.965 INFO:tasks.workunit.client.0.trial144.stderr:pool 'testpool' created 2026-02-19T12:48:05.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:48:06.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:557: TEST_auto_repair_bluestore_scrub: ceph osd pool set testpool size 2 2026-02-19T12:48:07.160 INFO:tasks.workunit.client.0.trial144.stderr:set pool 1 size to 2 2026-02-19T12:48:07.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:558: TEST_auto_repair_bluestore_scrub: wait_for_clean 2026-02-19T12:48:07.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:48:07.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:48:07.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:48:07.172 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:48:07.172 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:48:07.172 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:48:07.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:48:07.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:48:07.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:48:07.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:48:07.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:48:07.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:48:07.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:48:07.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:48:07.285 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:48:07.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:48:07.449 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:48:07.449 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:48:07.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:48:07.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:48:07.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:48:07.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T12:48:07.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T12:48:07.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T12:48:07.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:48:07.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:48:07.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672963 2026-02-19T12:48:07.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672963 2026-02-19T12:48:07.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-42949672963' 2026-02-19T12:48:07.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:48:07.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:48:07.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542146 2026-02-19T12:48:07.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542146 2026-02-19T12:48:07.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-42949672963 2-60129542146' 2026-02-19T12:48:07.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:48:07.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T12:48:07.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:48:07.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:48:07.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T12:48:07.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:48:07.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T12:48:07.670 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T12:48:07.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T12:48:07.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:48:07.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T12:48:07.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:48:08.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:48:08.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:48:08.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T12:48:08.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:48:09.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:48:09.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:48:10.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836483 2026-02-19T12:48:10.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:48:10.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672963 2026-02-19T12:48:10.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:48:10.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:48:10.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672963 2026-02-19T12:48:10.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:48:10.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672963 2026-02-19T12:48:10.163 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672963 2026-02-19T12:48:10.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672963' 2026-02-19T12:48:10.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:48:10.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672963 -lt 42949672963 2026-02-19T12:48:10.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:48:10.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542146 2026-02-19T12:48:10.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:48:10.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:48:10.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542146 2026-02-19T12:48:10.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:48:10.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542146 2026-02-19T12:48:10.325 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542146 2026-02-19T12:48:10.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-19T12:48:10.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:48:10.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542146 -lt 60129542146 2026-02-19T12:48:10.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:48:10.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:48:10.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:48:10.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:48:10.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:48:10.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:48:10.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:48:10.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:48:10.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:48:10.673 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:48:10.673 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:48:10.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T12:48:10.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:48:10.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:48:10.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:48:11.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T12:48:11.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-19T12:48:11.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T12:48:11.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-19T12:48:11.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:48:11.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T12:48:11.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:48:11.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:48:11.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:48:11.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:48:11.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:48:11.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:48:11.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:48:11.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:48:11.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T12:48:11.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:48:11.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:48:11.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:48:11.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T12:48:11.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-19T12:48:11.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:48:11.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:48:11.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:48:11.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:48:11.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:48:11.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:48:11.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:48:11.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:48:11.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:48:11.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T12:48:11.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:48:11.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T12:48:11.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:48:11.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:48:11.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:48:11.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:48:11.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:48:11.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:48:11.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:48:11.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:48:12.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T12:48:12.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:48:12.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:48:12.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:48:12.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T12:48:12.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-19T12:48:12.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:48:12.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:48:12.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:48:12.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:48:12.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:48:12.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:48:12.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:48:12.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:48:12.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:48:12.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-19T12:48:12.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:48:12.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-19T12:48:12.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:48:12.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:48:12.817 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:48:12.817 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:48:12.817 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:48:12.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:48:12.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:48:12.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:48:12.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T12:48:12.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:48:12.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:48:12.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:48:13.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T12:48:13.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-19T12:48:13.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:48:13.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:48:13.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:48:13.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:48:13.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:48:13.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:48:13.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:48:13.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:48:13.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:48:13.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 13 )) 2026-02-19T12:48:13.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:48:13.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-19T12:48:14.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:48:14.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:48:14.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:48:14.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:48:14.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:48:14.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:48:14.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:48:14.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:48:14.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:48:14.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:48:14.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:48:14.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:48:14.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:48:14.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:48:14.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:48:14.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_scrub: local payload=ABCDEF 2026-02-19T12:48:14.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:562: TEST_auto_repair_bluestore_scrub: echo ABCDEF 2026-02-19T12:48:14.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:563: TEST_auto_repair_bluestore_scrub: rados --pool testpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-19T12:48:14.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:567: TEST_auto_repair_bluestore_scrub: get_not_primary testpool SOMETHING 2026-02-19T12:48:14.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=testpool 2026-02-19T12:48:14.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:48:14.535 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary testpool SOMETHING 2026-02-19T12:48:14.535 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:48:14.535 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:48:14.535 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:48:14.535 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:48:14.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:48:14.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:48:14.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:48:14.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:567: TEST_auto_repair_bluestore_scrub: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T12:48:14.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:48:14.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:48:14.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:48:14.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:48:14.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T12:48:14.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:48:14.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:48:14.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:48:14.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:48:14.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:48:14.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:48:14.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:48:14.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:48:14.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:48:14.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:48:15.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:48:15.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T12:48:15.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:48:15.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:48:15.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:48:15.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:48:15.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:48:15.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-19T12:48:15.371 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-19T12:48:15.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-19T12:48:15.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:48:15.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:48:15.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:48:15.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:48:15.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:48:15.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:48:15.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:48:15.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:48:15.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:48:15.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:48:15.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:48:15.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:48:15.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:48:15.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:48:15.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:48:15.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:48:15.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:48:15.499 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:48:15.499 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:48:15.499 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:48:15.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:48:15.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:48:15.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:48:15.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:48:15.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:48:15.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:48:15.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:48:15.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:48:15.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:48:15.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:48:15.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-19T12:48:15.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:48:15.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:48:15.502 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:48:15.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-19T12:48:15.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:48:15.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:48:15.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:48:15.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:48:15.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:48:15.525 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:15.523+0000 7fe4bdfaaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:15.527 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:15.523+0000 7fe4bdfaaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:15.528 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:15.527+0000 7fe4bdfaaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:15.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:48:15.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:48:15.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:48:15.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:48:15.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:48:15.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:48:15.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:48:15.661 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:48:15.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:48:15.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:48:15.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:48:15.869 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:15.867+0000 7fe4bdfaaa40 -1 Falling back to public interface 2026-02-19T12:48:16.433 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:16.431+0000 7fe4bdfaaa40 -1 osd.0 19 log_to_monitors true 2026-02-19T12:48:16.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:48:16.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:48:16.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:48:16.825 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:48:16.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:48:16.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:48:16.981 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 23 up_thru 0 down_at 19 last_clean_interval [5,19) [v2:127.0.0.1:6802/2652340779,v1:127.0.0.1:6803/2652340779] [v2:127.0.0.1:6804/2652340779,v1:127.0.0.1:6805/2652340779] exists,up 42b4dfbd-280e-4a0a-9f9d-28b259a62af8 2026-02-19T12:48:16.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:48:16.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:48:16.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:48:16.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:48:16.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:48:16.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:48:16.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:48:16.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:48:16.983 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:48:16.983 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:48:16.983 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:48:16.983 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:48:16.983 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:48:17.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:48:17.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:48:17.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:48:17.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:48:17.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:48:17.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:48:17.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:48:17.251 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:48:17.251 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:48:17.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:48:17.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:48:17.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:48:17.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247810 2026-02-19T12:48:17.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247810 2026-02-19T12:48:17.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247810' 2026-02-19T12:48:17.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:48:17.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:48:17.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672966 2026-02-19T12:48:17.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672966 2026-02-19T12:48:17.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247810 1-42949672966' 2026-02-19T12:48:17.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:48:17.398 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:48:17.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542149 2026-02-19T12:48:17.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542149 2026-02-19T12:48:17.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247810 1-42949672966 2-60129542149' 2026-02-19T12:48:17.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:48:17.473 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-98784247810 2026-02-19T12:48:17.473 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:48:17.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:48:17.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-98784247810 2026-02-19T12:48:17.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:48:17.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=98784247810 2026-02-19T12:48:17.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 98784247810' 2026-02-19T12:48:17.477 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 98784247810 2026-02-19T12:48:17.477 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:48:17.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 98784247810 2026-02-19T12:48:17.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:48:18.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:48:18.637 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:48:18.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 98784247810 2026-02-19T12:48:18.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:48:19.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:48:19.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:48:19.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247810 -lt 98784247810 2026-02-19T12:48:19.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:48:19.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672966 2026-02-19T12:48:19.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:48:19.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:48:19.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672966 2026-02-19T12:48:19.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:48:19.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672966 2026-02-19T12:48:19.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-19T12:48:19.958 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 42949672966 2026-02-19T12:48:19.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:48:20.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672966 -lt 42949672966 2026-02-19T12:48:20.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:48:20.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542149 2026-02-19T12:48:20.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:48:20.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:48:20.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542149 2026-02-19T12:48:20.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:48:20.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542149 2026-02-19T12:48:20.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542149' 2026-02-19T12:48:20.132 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542149 2026-02-19T12:48:20.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:48:20.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542149 -lt 60129542149 2026-02-19T12:48:20.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:48:20.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:48:20.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:48:20.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:48:20.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:48:20.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:48:20.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:48:20.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:48:20.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:48:20.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:48:20.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:48:20.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:48:20.649 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:48:20.649 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:48:20.649 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:48:20.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:48:20.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:48:20.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:48:20.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:569: TEST_auto_repair_bluestore_scrub: get_pg testpool SOMETHING 2026-02-19T12:48:20.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=testpool 2026-02-19T12:48:20.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:48:20.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map testpool SOMETHING 2026-02-19T12:48:20.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:48:21.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:569: TEST_auto_repair_bluestore_scrub: local pgid=1.0 2026-02-19T12:48:21.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:570: TEST_auto_repair_bluestore_scrub: get_primary testpool SOMETHING 2026-02-19T12:48:21.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:48:21.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:48:21.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:48:21.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:48:21.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:570: TEST_auto_repair_bluestore_scrub: local primary=1 2026-02-19T12:48:21.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:571: TEST_auto_repair_bluestore_scrub: get_last_scrub_stamp 1.0 2026-02-19T12:48:21.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:48:21.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:48:21.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:48:21.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:48:21.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:571: TEST_auto_repair_bluestore_scrub: local last_scrub_stamp=2026-02-19T12:48:05.964292+0000 2026-02-19T12:48:21.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:572: TEST_auto_repair_bluestore_scrub: ceph tell 1.0 schedule-scrub 2026-02-19T12:48:21.392 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T12:48:21.392 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T12:48:21.392 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T12:48:21.392 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-17T12:48:21.392491+0000" 2026-02-19T12:48:21.392 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T12:48:21.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:575: TEST_auto_repair_bluestore_scrub: wait_for_scrub 1.0 2026-02-19T12:48:05.964292+0000 2026-02-19T12:48:21.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T12:48:21.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:48:05.964292+0000 2026-02-19T12:48:21.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:48:21.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:48:21.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:48:21.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:48:21.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:48:21.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:48:21.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:48:21.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:48:21.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:48:05.964292+0000 '>' 2026-02-19T12:48:05.964292+0000 2026-02-19T12:48:21.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:48:22.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:48:22.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:48:22.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:48:22.562 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:48:22.562 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:48:22.562 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:48:22.562 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:48:22.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:48:05.964292+0000 '>' 2026-02-19T12:48:05.964292+0000 2026-02-19T12:48:22.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:48:23.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:48:23.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:48:23.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:48:23.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:48:23.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:48:23.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:48:23.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:48:23.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T12:48:21.392491+0000 '>' 2026-02-19T12:48:05.964292+0000 2026-02-19T12:48:23.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:48:24.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:48:24.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:48:24.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:48:24.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:48:24.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:48:24.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:48:24.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:48:25.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T12:48:21.392491+0000 '>' 2026-02-19T12:48:05.964292+0000 2026-02-19T12:48:25.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:48:26.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:48:26.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:48:26.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:48:26.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:48:26.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:48:26.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:48:26.054 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:48:26.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T12:48:21.392491+0000 '>' 2026-02-19T12:48:05.964292+0000 2026-02-19T12:48:26.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:48:27.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:48:27.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:48:27.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T12:48:27.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:48:27.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:48:27.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:48:27.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:48:27.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:48:23.273177+0000 '>' 2026-02-19T12:48:05.964292+0000 2026-02-19T12:48:27.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:48:27.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:576: TEST_auto_repair_bluestore_scrub: ceph pg dump pgs 2026-02-19T12:48:27.514 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:48:27.514 INFO:tasks.workunit.client.0.trial144.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean 2026-02-19T12:48:23.274937+0000 18'1 24:66 [1,0] 1 [1,0] 1 18'1 2026-02-19T12:48:23.273177+0000 18'1 2026-02-19T12:48:23.273177+0000 0 1 periodic scrub scheduled @ 2026-02-20T12:48:23.273177+0000 1 0 2026-02-19T12:48:27.514 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:48:27.514 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:48:27.515 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:48:27.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:578: TEST_auto_repair_bluestore_scrub: sleep 2 2026-02-19T12:48:29.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:579: TEST_auto_repair_bluestore_scrub: ceph pg dump pgs 2026-02-19T12:48:29.678 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:48:29.678 INFO:tasks.workunit.client.0.trial144.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean 2026-02-19T12:48:23.274937+0000 18'1 24:66 [1,0] 1 [1,0] 1 18'1 2026-02-19T12:48:23.273177+0000 18'1 2026-02-19T12:48:23.273177+0000 0 1 periodic scrub scheduled @ 2026-02-20T12:48:23.273177+0000 1 0 2026-02-19T12:48:29.679 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:48:29.679 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:48:29.679 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:48:29.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:580: TEST_auto_repair_bluestore_scrub: sleep 2 2026-02-19T12:48:31.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:581: TEST_auto_repair_bluestore_scrub: ceph pg dump pgs 2026-02-19T12:48:31.835 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:48:31.835 INFO:tasks.workunit.client.0.trial144.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean 2026-02-19T12:48:23.274937+0000 18'1 24:66 [1,0] 1 [1,0] 1 18'1 2026-02-19T12:48:23.273177+0000 18'1 2026-02-19T12:48:23.273177+0000 0 1 periodic scrub scheduled @ 2026-02-20T12:48:23.273177+0000 1 0 2026-02-19T12:48:31.835 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:48:31.835 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:48:31.835 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:48:31.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:582: TEST_auto_repair_bluestore_scrub: sleep 5 2026-02-19T12:48:36.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:583: TEST_auto_repair_bluestore_scrub: wait_for_clean 2026-02-19T12:48:36.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:48:36.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:48:36.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:48:36.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:48:36.848 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:48:36.848 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:48:36.849 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:48:36.849 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:48:36.849 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:48:36.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:48:36.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:48:36.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:48:36.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:48:36.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:48:36.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:48:37.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:48:37.120 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:48:37.120 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:48:37.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:48:37.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:48:37.120 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:48:37.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247815 2026-02-19T12:48:37.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247815 2026-02-19T12:48:37.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247815' 2026-02-19T12:48:37.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:48:37.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:48:37.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672971 2026-02-19T12:48:37.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672971 2026-02-19T12:48:37.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247815 1-42949672971' 2026-02-19T12:48:37.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:48:37.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:48:37.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542154 2026-02-19T12:48:37.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542154 2026-02-19T12:48:37.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247815 1-42949672971 2-60129542154' 2026-02-19T12:48:37.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:48:37.341 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-98784247815 2026-02-19T12:48:37.341 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:48:37.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:48:37.343 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-98784247815 2026-02-19T12:48:37.343 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:48:37.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=98784247815 2026-02-19T12:48:37.344 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 98784247815 2026-02-19T12:48:37.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 98784247815' 2026-02-19T12:48:37.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:48:37.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247814 -lt 98784247815 2026-02-19T12:48:37.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:48:38.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:48:38.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:48:38.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247814 -lt 98784247815 2026-02-19T12:48:38.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:48:39.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:48:39.673 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:48:39.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247815 -lt 98784247815 2026-02-19T12:48:39.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:48:39.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672971 2026-02-19T12:48:39.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:48:39.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:48:39.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672971 2026-02-19T12:48:39.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:48:39.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672971 2026-02-19T12:48:39.836 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672971 2026-02-19T12:48:39.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672971' 2026-02-19T12:48:39.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:48:40.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672971 -lt 42949672971 2026-02-19T12:48:40.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:48:40.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542154 2026-02-19T12:48:40.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:48:40.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:48:40.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542154 2026-02-19T12:48:40.003 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:48:40.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542154 2026-02-19T12:48:40.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542154' 2026-02-19T12:48:40.004 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542154 2026-02-19T12:48:40.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:48:40.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542154 -lt 60129542154 2026-02-19T12:48:40.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:48:40.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:48:40.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:48:40.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:48:40.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:48:40.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:48:40.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:48:40.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:48:40.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:48:40.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:48:40.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:48:40.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:48:40.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:48:40.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:48:40.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:48:40.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:48:40.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:48:40.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:48:40.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:584: TEST_auto_repair_bluestore_scrub: ceph pg dump pgs 2026-02-19T12:48:40.856 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:48:40.856 INFO:tasks.workunit.client.0.trial144.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean 2026-02-19T12:48:23.274937+0000 18'1 24:66 [1,0] 1 [1,0] 1 18'1 2026-02-19T12:48:23.273177+0000 18'1 2026-02-19T12:48:23.273177+0000 0 1 periodic scrub scheduled @ 2026-02-20T12:48:23.273177+0000 1 0 2026-02-19T12:48:40.856 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:48:40.856 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:48:40.857 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:48:40.868 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:587: TEST_auto_repair_bluestore_scrub: get_not_primary testpool SOMETHING 2026-02-19T12:48:40.868 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=testpool 2026-02-19T12:48:40.868 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:48:40.869 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary testpool SOMETHING 2026-02-19T12:48:40.869 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:48:40.869 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:48:40.869 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:48:40.869 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:48:41.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:48:41.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:48:41.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:48:41.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:587: TEST_auto_repair_bluestore_scrub: objectstore_tool td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-19T12:48:41.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:48:41.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:48:41.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:48:41.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:48:41.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-19T12:48:41.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:48:41.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:48:41.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:48:41.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:48:41.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:48:41.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:48:41.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:48:41.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:48:41.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:48:41.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:48:41.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:48:41.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-19T12:48:41.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:48:41.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:48:41.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:48:41.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:48:41.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:48:41.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING list-attrs 2026-02-19T12:48:41.558 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:48:41.558 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:48:41.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-19T12:48:41.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:48:41.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:48:41.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:48:41.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:48:41.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:48:41.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:48:41.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:48:41.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:48:41.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:48:41.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:48:41.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:48:41.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:48:41.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:48:41.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:48:41.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:48:41.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:48:41.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:48:41.652 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:48:41.652 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:48:41.652 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:48:41.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:48:41.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:48:41.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:48:41.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:48:41.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:48:41.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:48:41.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:48:41.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:48:41.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:48:41.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:48:41.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-19T12:48:41.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:48:41.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:48:41.654 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:48:41.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-19T12:48:41.654 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:48:41.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:48:41.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:48:41.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:48:41.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:48:41.670 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:41.666+0000 7f8a84be6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:41.671 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:41.666+0000 7f8a84be6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:41.672 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:41.670+0000 7f8a84be6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:41.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:48:41.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:48:41.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:48:41.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:48:41.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:48:41.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:48:41.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:48:41.810 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:48:41.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:48:41.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:48:41.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:48:42.049 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:42.046+0000 7f8a84be6a40 -1 Falling back to public interface 2026-02-19T12:48:42.607 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:42.602+0000 7f8a84be6a40 -1 osd.0 24 log_to_monitors true 2026-02-19T12:48:42.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:48:42.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:48:42.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:48:42.969 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:48:42.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:48:42.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:48:43.127 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 28 up_thru 0 down_at 25 last_clean_interval [23,24) [v2:127.0.0.1:6802/3793487347,v1:127.0.0.1:6803/3793487347] [v2:127.0.0.1:6804/3793487347,v1:127.0.0.1:6805/3793487347] exists,up 42b4dfbd-280e-4a0a-9f9d-28b259a62af8 2026-02-19T12:48:43.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:48:43.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:48:43.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:48:43.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:48:43.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:48:43.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:48:43.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:48:43.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:48:43.128 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:48:43.128 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:48:43.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:48:43.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:48:43.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:48:43.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:48:43.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:48:43.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:48:43.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:48:43.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:48:43.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:48:43.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:48:43.402 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:48:43.402 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:48:43.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:48:43.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:48:43.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:48:43.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084290 2026-02-19T12:48:43.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084290 2026-02-19T12:48:43.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-120259084290' 2026-02-19T12:48:43.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:48:43.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:48:43.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672973 2026-02-19T12:48:43.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672973 2026-02-19T12:48:43.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-120259084290 1-42949672973' 2026-02-19T12:48:43.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:48:43.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:48:43.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542156 2026-02-19T12:48:43.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542156 2026-02-19T12:48:43.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-120259084290 1-42949672973 2-60129542156' 2026-02-19T12:48:43.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:48:43.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-120259084290 2026-02-19T12:48:43.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:48:43.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:48:43.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-120259084290 2026-02-19T12:48:43.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:48:43.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=120259084290 2026-02-19T12:48:43.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 120259084290' 2026-02-19T12:48:43.622 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 120259084290 2026-02-19T12:48:43.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:48:43.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 120259084290 2026-02-19T12:48:43.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:48:44.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:48:44.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:48:44.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 120259084290 2026-02-19T12:48:44.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:48:45.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:48:45.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:48:46.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084290 -lt 120259084290 2026-02-19T12:48:46.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:48:46.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672973 2026-02-19T12:48:46.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:48:46.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:48:46.098 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672973 2026-02-19T12:48:46.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:48:46.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672973 2026-02-19T12:48:46.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672973' 2026-02-19T12:48:46.100 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 42949672973 2026-02-19T12:48:46.101 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:48:46.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672973 -lt 42949672973 2026-02-19T12:48:46.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:48:46.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542156 2026-02-19T12:48:46.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:48:46.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:48:46.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542156 2026-02-19T12:48:46.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:48:46.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542156 2026-02-19T12:48:46.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542156' 2026-02-19T12:48:46.263 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542156 2026-02-19T12:48:46.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:48:46.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542156 -lt 60129542156 2026-02-19T12:48:46.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:48:46.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:48:46.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:48:46.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:48:46.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:48:46.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:48:46.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:48:46.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:48:46.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:48:46.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:48:46.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:48:46.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:48:46.771 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:48:46.771 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:48:46.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:48:46.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:48:46.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:48:46.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:48:46.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:588: TEST_auto_repair_bluestore_scrub: rados --pool testpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:48:46.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:589: TEST_auto_repair_bluestore_scrub: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:48:46.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:590: TEST_auto_repair_bluestore_scrub: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-19T12:48:46.986 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:48:22.310+0000 7f09c756f640 10 osd.1 pg_epoch: 24 pg[1.0( v 18'1 (0'0,18'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=18'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing(periodic-regular,deep-scrub-on-error)] scrubber: scrub_finish before flags: DEEP_SCRUB_ON_ERROR. repair state: no-repair. deep_scrub_on_error: 1 2026-02-19T12:48:46.986 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:48:22.310+0000 7f09c756f640 10 osd.1 pg_epoch: 24 pg[1.0( v 18'1 (0'0,18'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=18'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing(periodic-regular,deep-scrub-on-error)] scrubber: scrub_finish: will initiate a deep scrub to fix 1 errors 2026-02-19T12:48:46.986 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:48:22.310+0000 7f09c756f640 10 osd.1 pg_epoch: 24 pg[1.0( v 18'1 (0'0,18'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=18'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing(periodic-regular)] scrubber: _scrub_finish info stats: valid m_is_repair: 0 2026-02-19T12:48:46.986 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:48:23.270+0000 7f09cb577640 10 osd.1 pg_epoch: 24 pg[1.0( v 18'1 (0'0,18'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=18'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent+repair(repairing,auto-repair)] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: repair. deep_scrub_on_error: 0 2026-02-19T12:48:46.987 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:48:23.270+0000 7f09cb577640 10 osd.1 pg_epoch: 24 pg[1.0( v 18'1 (0'0,18'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=18'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent+repair(repairing,auto-repair)] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-19T12:48:46.987 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:48:23.270+0000 7f09cb577640 15 osd.1 pg_epoch: 24 pg[1.0( v 18'1 (0'0,18'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=18'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+inconsistent+repair(repairing,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish: 1 errors. 1 errors fixed 2026-02-19T12:48:46.987 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:48:23.270+0000 7f09cb577640 20 osd.1 pg_epoch: 24 pg[1.0( v 18'1 (0'0,18'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=18'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+inconsistent+repair(repairing,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish All may be fixed 2026-02-19T12:48:46.987 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:48:23.270+0000 7f09cb577640 19 osd.1 pg_epoch: 24 pg[1.0( v 18'1 (0'0,18'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=18'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+inconsistent+repair(repairing,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-19T12:48:46.987 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:593: TEST_auto_repair_bluestore_scrub: ceph pg 1.0 query 2026-02-19T12:48:46.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:593: TEST_auto_repair_bluestore_scrub: jq .info.stats.stat_sum.num_objects_repaired 2026-02-19T12:48:47.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:593: TEST_auto_repair_bluestore_scrub: COUNT=1 2026-02-19T12:48:47.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:594: TEST_auto_repair_bluestore_scrub: test 1 = 1 2026-02-19T12:48:47.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T12:48:47.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:48:47.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:48:47.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:48:47.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:48:47.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:48:47.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:48:47.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:48:47.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:48:47.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:48:47.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:48:47.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:48:47.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:48:47.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:48:47.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:48:47.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:48:47.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:48:47.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:48:47.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:48:47.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:48:47.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:48:47.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:48:47.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:48:47.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:48:47.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:48:47.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:48:47.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:48:47.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:48:47.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:48:47.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T12:48:47.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T12:48:47.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T12:48:47.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T12:48:47.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:48:47.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:48:47.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:48:47.193 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:48:47.193 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:48:47.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:48:47.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:48:47.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:48:47.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:48:47.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:48:47.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:48:47.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:48:47.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:48:47.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:48:47.200 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:48:47.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:48:47.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:48:47.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:48:47.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:48:47.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:48:47.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:48:47.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:48:47.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:48:47.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:48:47.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:48:47.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:48:47.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:48:47.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:48:47.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T12:48:47.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:48:47.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:48:47.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:48:47.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T12:48:47.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:48:47.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T12:48:47.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:48:47.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T12:48:47.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_bluestore_tag td/osd-scrub-repair 2026-02-19T12:48:47.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:443: TEST_auto_repair_bluestore_tag: local dir=td/osd-scrub-repair 2026-02-19T12:48:47.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:444: TEST_auto_repair_bluestore_tag: local poolname=testpool 2026-02-19T12:48:47.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:447: TEST_auto_repair_bluestore_tag: run_mon td/osd-scrub-repair a 2026-02-19T12:48:47.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T12:48:47.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T12:48:47.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T12:48:47.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T12:48:47.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T12:48:47.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T12:48:47.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T12:48:47.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:48:47.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:48:47.234 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:48:47.234 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:48:47.234 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:48:47.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:48:47.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T12:48:47.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T12:48:47.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T12:48:47.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:48:47.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:48:47.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T12:48:47.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T12:48:47.264 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:48:47.264 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:48:47.264 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:48:47.264 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:48:47.264 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:48:47.264 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:48:47.265 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:48:47.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:48:47.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T12:48:47.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T12:48:47.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:48:47.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:48:47.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T12:48:47.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T12:48:47.321 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:48:47.321 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:48:47.321 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:48:47.321 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:48:47.321 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:48:47.321 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:48:47.321 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:48:47.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:48:47.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T12:48:47.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:448: TEST_auto_repair_bluestore_tag: run_mgr td/osd-scrub-repair x 2026-02-19T12:48:47.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T12:48:47.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T12:48:47.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T12:48:47.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T12:48:47.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T12:48:47.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T12:48:47.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T12:48:47.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:48:47.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:48:47.469 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:48:47.470 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:48:47.470 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:48:47.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:48:47.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:48:47.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:48:47.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:455: TEST_auto_repair_bluestore_tag: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-19T12:48:47.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:456: TEST_auto_repair_bluestore_tag: seq 0 2 2026-02-19T12:48:47.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:456: TEST_auto_repair_bluestore_tag: for id in $(seq 0 2) 2026-02-19T12:48:47.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:457: TEST_auto_repair_bluestore_tag: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-19T12:48:47.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:48:47.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:48:47.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T12:48:47.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:48:47.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:48:47.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:48:47.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:48:47.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:48:47.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:48:47.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:48:47.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:48:47.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:48:47.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:48:47.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:48:47.494 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:48:47.494 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:48:47.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:48:47.495 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:48:47.495 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:48:47.495 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:48:47.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:48:47.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:48:47.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:48:47.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:48:47.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:48:47.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:48:47.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:48:47.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:48:47.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:48:47.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:48:47.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:48:47.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:48:47.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-19T12:48:47.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:48:47.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:48:47.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ac4b7d16-5de1-420d-9846-ba20aca6d6d7 2026-02-19T12:48:47.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ac4b7d16-5de1-420d-9846-ba20aca6d6d7' 2026-02-19T12:48:47.497 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 ac4b7d16-5de1-420d-9846-ba20aca6d6d7 2026-02-19T12:48:47.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:48:47.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCvBpdpASo5HhAA+943zqyqYFBMKGfWnPjh4g== 2026-02-19T12:48:47.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCvBpdpASo5HhAA+943zqyqYFBMKGfWnPjh4g=="}' 2026-02-19T12:48:47.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ac4b7d16-5de1-420d-9846-ba20aca6d6d7 -i td/osd-scrub-repair/0/new.json 2026-02-19T12:48:47.598 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:48:47.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T12:48:47.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq --mkfs --key AQCvBpdpASo5HhAA+943zqyqYFBMKGfWnPjh4g== --osd-uuid ac4b7d16-5de1-420d-9846-ba20aca6d6d7 2026-02-19T12:48:47.624 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:47.622+0000 7f6850ad9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:47.625 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:47.622+0000 7f6850ad9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:47.626 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:47.622+0000 7f6850ad9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:47.627 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:47.622+0000 7f6850ad9a40 -1 bdev(0x55a7a6b74c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:48:47.627 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:47.622+0000 7f6850ad9a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T12:48:48.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T12:48:48.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:48:48.340 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:48:48.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T12:48:48.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:48:48.542 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:48:48.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T12:48:48.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-19T12:48:48.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:48:48.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:48:48.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:48:48.568 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:48.566+0000 7f9b69a0da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:48.569 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:48.566+0000 7f9b69a0da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:48.571 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:48.566+0000 7f9b69a0da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:48.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T12:48:48.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:48:48.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:48:48.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:48:48.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:48:48.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:48:48.703 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:48:48.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:48:48.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:48:48.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:48:48.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:48:48.937 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:48.934+0000 7f9b69a0da40 -1 Falling back to public interface 2026-02-19T12:48:49.493 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:49.490+0000 7f9b69a0da40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:48:49.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:48:49.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:48:49.862 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:48:49.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:48:49.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:48:49.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:48:50.019 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2631085711,v1:127.0.0.1:6803/2631085711] [v2:127.0.0.1:6804/2631085711,v1:127.0.0.1:6805/2631085711] exists,up ac4b7d16-5de1-420d-9846-ba20aca6d6d7 2026-02-19T12:48:50.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:48:50.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:48:50.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:48:50.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:456: TEST_auto_repair_bluestore_tag: for id in $(seq 0 2) 2026-02-19T12:48:50.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:457: TEST_auto_repair_bluestore_tag: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-19T12:48:50.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:48:50.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:48:50.022 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:48:50.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:48:50.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:48:50.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:48:50.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:48:50.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:48:50.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:48:50.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:48:50.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:48:50.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:48:50.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:48:50.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:48:50.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:48:50.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-19T12:48:50.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:48:50.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:48:50.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f469c162-94e7-4c9b-8ad5-36f19f1843b9 2026-02-19T12:48:50.025 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 f469c162-94e7-4c9b-8ad5-36f19f1843b9 2026-02-19T12:48:50.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 f469c162-94e7-4c9b-8ad5-36f19f1843b9' 2026-02-19T12:48:50.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:48:50.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCyBpdpn3w6AhAA9WhcjI+44z4JYve+O9wDDg== 2026-02-19T12:48:50.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCyBpdpn3w6AhAA9WhcjI+44z4JYve+O9wDDg=="}' 2026-02-19T12:48:50.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f469c162-94e7-4c9b-8ad5-36f19f1843b9 -i td/osd-scrub-repair/1/new.json 2026-02-19T12:48:50.198 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:48:50.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T12:48:50.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq --mkfs --key AQCyBpdpn3w6AhAA9WhcjI+44z4JYve+O9wDDg== --osd-uuid f469c162-94e7-4c9b-8ad5-36f19f1843b9 2026-02-19T12:48:50.225 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:50.222+0000 7f9b04282a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:50.227 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:50.222+0000 7f9b04282a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:50.228 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:50.226+0000 7f9b04282a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:50.229 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:50.226+0000 7f9b04282a40 -1 bdev(0x5574fe092c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:48:50.229 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:50.226+0000 7f9b04282a40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T12:48:50.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T12:48:50.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:48:50.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T12:48:50.896 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:48:50.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:48:51.089 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:48:51.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T12:48:51.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-19T12:48:51.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:48:51.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:48:51.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:48:51.115 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:51.110+0000 7fa65d145a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:51.116 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:51.114+0000 7fa65d145a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:51.117 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:51.114+0000 7fa65d145a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:51.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T12:48:51.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:48:51.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:48:51.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:48:51.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:48:51.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:48:51.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:48:51.250 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:48:51.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:48:51.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:48:51.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:48:51.509 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:51.506+0000 7fa65d145a40 -1 Falling back to public interface 2026-02-19T12:48:52.105 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:52.102+0000 7fa65d145a40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:48:52.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:48:52.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:48:52.406 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:48:52.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:48:52.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:48:52.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:48:52.560 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/437015068,v1:127.0.0.1:6811/437015068] [v2:127.0.0.1:6812/437015068,v1:127.0.0.1:6813/437015068] exists,up f469c162-94e7-4c9b-8ad5-36f19f1843b9 2026-02-19T12:48:52.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:48:52.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:48:52.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:48:52.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:456: TEST_auto_repair_bluestore_tag: for id in $(seq 0 2) 2026-02-19T12:48:52.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:457: TEST_auto_repair_bluestore_tag: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-19T12:48:52.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:48:52.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:48:52.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T12:48:52.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:48:52.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:48:52.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:48:52.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:48:52.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:48:52.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:48:52.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:48:52.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:48:52.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:48:52.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:48:52.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:48:52.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:48:52.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:48:52.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:48:52.565 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:48:52.566 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:48:52.566 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:48:52.566 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:48:52.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:48:52.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:48:52.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:48:52.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:48:52.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:48:52.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:48:52.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:48:52.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:48:52.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:48:52.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:48:52.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:48:52.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-19T12:48:52.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:48:52.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:48:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bffcde66-ac34-42b5-add4-acb508b028d6 2026-02-19T12:48:52.568 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 bffcde66-ac34-42b5-add4-acb508b028d6 2026-02-19T12:48:52.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 bffcde66-ac34-42b5-add4-acb508b028d6' 2026-02-19T12:48:52.568 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:48:52.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC0Bpdp91R0IhAAho1VjUSXNqsBK0MZ70b/tg== 2026-02-19T12:48:52.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC0Bpdp91R0IhAAho1VjUSXNqsBK0MZ70b/tg=="}' 2026-02-19T12:48:52.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bffcde66-ac34-42b5-add4-acb508b028d6 -i td/osd-scrub-repair/2/new.json 2026-02-19T12:48:52.741 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:48:52.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T12:48:52.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq --mkfs --key AQC0Bpdp91R0IhAAho1VjUSXNqsBK0MZ70b/tg== --osd-uuid bffcde66-ac34-42b5-add4-acb508b028d6 2026-02-19T12:48:52.770 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:52.766+0000 7f741b4c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:52.772 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:52.770+0000 7f741b4c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:52.773 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:52.770+0000 7f741b4c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:52.774 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:52.770+0000 7f741b4c5a40 -1 bdev(0x5649975acc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T12:48:52.774 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:52.770+0000 7f741b4c5a40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T12:48:53.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T12:48:53.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:48:53.390 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T12:48:53.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T12:48:53.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:48:53.595 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T12:48:53.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T12:48:53.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-19T12:48:53.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:48:53.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:48:53.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:48:53.622 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:53.618+0000 7f094a7b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:53.624 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:53.622+0000 7f094a7b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:53.625 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:53.622+0000 7f094a7b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:48:53.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T12:48:53.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:48:53.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:48:53.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:48:53.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:48:53.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:48:53.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:48:53.752 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:48:53.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:48:53.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:48:53.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:48:54.037 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:54.034+0000 7f094a7b2a40 -1 Falling back to public interface 2026-02-19T12:48:54.725 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:48:54.722+0000 7f094a7b2a40 -1 osd.2 0 log_to_monitors true 2026-02-19T12:48:54.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:48:54.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:48:54.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:48:54.910 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:48:54.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:48:54.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:48:55.066 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3571097892,v1:127.0.0.1:6819/3571097892] [v2:127.0.0.1:6820/3571097892,v1:127.0.0.1:6821/3571097892] exists,up bffcde66-ac34-42b5-add4-acb508b028d6 2026-02-19T12:48:55.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:48:55.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:48:55.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:48:55.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:460: TEST_auto_repair_bluestore_tag: create_pool testpool 1 1 2026-02-19T12:48:55.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create testpool 1 1 2026-02-19T12:48:55.287 INFO:tasks.workunit.client.0.trial144.stderr:pool 'testpool' created 2026-02-19T12:48:55.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:48:56.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:461: TEST_auto_repair_bluestore_tag: ceph osd pool set testpool size 2 2026-02-19T12:48:56.496 INFO:tasks.workunit.client.0.trial144.stderr:set pool 1 size to 2 2026-02-19T12:48:56.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:462: TEST_auto_repair_bluestore_tag: wait_for_clean 2026-02-19T12:48:56.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:48:56.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:48:56.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:48:56.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:48:56.508 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:48:56.508 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:48:56.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:48:56.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:48:56.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:48:56.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:48:56.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:48:56.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:48:56.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:48:56.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:48:56.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:48:56.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:48:56.771 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:48:56.771 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:48:56.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:48:56.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:48:56.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:48:56.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T12:48:56.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T12:48:56.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T12:48:56.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:48:56.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:48:56.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672962 2026-02-19T12:48:56.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672962 2026-02-19T12:48:56.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-42949672962' 2026-02-19T12:48:56.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:48:56.921 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:48:56.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542146 2026-02-19T12:48:56.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542146 2026-02-19T12:48:56.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-42949672962 2-60129542146' 2026-02-19T12:48:56.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:48:56.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T12:48:56.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:48:56.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:48:56.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T12:48:56.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:48:56.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T12:48:56.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T12:48:56.998 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T12:48:56.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:48:57.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T12:48:57.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:48:58.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:48:58.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:48:58.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-19T12:48:58.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:48:58.318 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672962 2026-02-19T12:48:58.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:48:58.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:48:58.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672962 2026-02-19T12:48:58.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:48:58.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672962 2026-02-19T12:48:58.323 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672962 2026-02-19T12:48:58.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672962' 2026-02-19T12:48:58.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:48:58.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672962 -lt 42949672962 2026-02-19T12:48:58.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:48:58.494 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542146 2026-02-19T12:48:58.494 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:48:58.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:48:58.496 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542146 2026-02-19T12:48:58.496 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:48:58.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542146 2026-02-19T12:48:58.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-19T12:48:58.498 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542146 2026-02-19T12:48:58.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:48:58.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542146 -lt 60129542146 2026-02-19T12:48:58.665 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:48:58.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:48:58.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:48:58.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:48:58.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:48:58.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:48:58.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:48:58.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:48:58.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:48:58.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:48:58.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:48:59.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:48:59.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:48:59.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:48:59.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:48:59.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:48:59.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:48:59.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:48:59.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:465: TEST_auto_repair_bluestore_tag: local payload=ABCDEF 2026-02-19T12:48:59.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:466: TEST_auto_repair_bluestore_tag: echo ABCDEF 2026-02-19T12:48:59.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:467: TEST_auto_repair_bluestore_tag: rados --pool testpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-19T12:48:59.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:471: TEST_auto_repair_bluestore_tag: get_not_primary testpool SOMETHING 2026-02-19T12:48:59.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=testpool 2026-02-19T12:48:59.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:48:59.233 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary testpool SOMETHING 2026-02-19T12:48:59.233 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:48:59.233 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:48:59.233 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:48:59.233 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:48:59.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:48:59.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:48:59.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:48:59.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:471: TEST_auto_repair_bluestore_tag: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T12:48:59.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:48:59.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:48:59.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:48:59.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:48:59.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T12:48:59.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:48:59.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:48:59.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:48:59.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:48:59.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:48:59.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:48:59.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:48:59.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:48:59.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:48:59.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:48:59.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:48:59.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T12:48:59.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:48:59.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:48:59.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:48:59.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:48:59.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:48:59.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-19T12:49:00.086 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-19T12:49:00.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-19T12:49:00.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:49:00.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:49:00.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:49:00.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:49:00.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:49:00.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:49:00.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:49:00.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:49:00.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:49:00.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:49:00.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:49:00.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:49:00.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:49:00.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:49:00.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:49:00.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:49:00.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:49:00.248 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:49:00.248 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:00.248 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:00.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:00.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:00.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:49:00.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:49:00.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:49:00.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:49:00.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:49:00.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:49:00.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:49:00.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:49:00.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-19T12:49:00.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:49:00.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:49:00.250 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:49:00.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-19T12:49:00.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:49:00.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:49:00.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:49:00.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:49:00.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:49:00.264 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:00.262+0000 7f4562b0fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:00.265 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:00.262+0000 7f4562b0fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:00.266 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:00.262+0000 7f4562b0fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:00.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:49:00.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:49:00.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:49:00.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:49:00.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:49:00.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:00.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:49:00.407 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:49:00.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:00.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:49:00.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:49:00.713 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:00.710+0000 7f4562b0fa40 -1 Falling back to public interface 2026-02-19T12:49:01.333 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:01.330+0000 7f4562b0fa40 -1 osd.0 20 log_to_monitors true 2026-02-19T12:49:01.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:49:01.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:01.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:49:01.562 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:49:01.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:01.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:49:01.724 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 24 up_thru 0 down_at 21 last_clean_interval [5,20) [v2:127.0.0.1:6802/1672364176,v1:127.0.0.1:6803/1672364176] [v2:127.0.0.1:6804/1672364176,v1:127.0.0.1:6805/1672364176] exists,up ac4b7d16-5de1-420d-9846-ba20aca6d6d7 2026-02-19T12:49:01.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:49:01.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:49:01.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:49:01.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:49:01.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:49:01.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:49:01.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:49:01.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:49:01.726 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:49:01.726 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:49:01.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:49:01.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:49:01.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:49:01.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:49:01.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:49:01.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:49:01.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:49:01.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:49:01.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:49:01.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:49:01.990 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:49:01.990 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:49:01.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:49:01.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:01.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:49:02.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215106 2026-02-19T12:49:02.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215106 2026-02-19T12:49:02.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-103079215106' 2026-02-19T12:49:02.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:02.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:49:02.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672964 2026-02-19T12:49:02.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672964 2026-02-19T12:49:02.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-103079215106 1-42949672964' 2026-02-19T12:49:02.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:02.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:49:02.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542148 2026-02-19T12:49:02.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542148 2026-02-19T12:49:02.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-103079215106 1-42949672964 2-60129542148' 2026-02-19T12:49:02.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:02.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-103079215106 2026-02-19T12:49:02.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:02.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:49:02.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-103079215106 2026-02-19T12:49:02.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:02.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=103079215106 2026-02-19T12:49:02.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 103079215106' 2026-02-19T12:49:02.211 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 103079215106 2026-02-19T12:49:02.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:02.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 103079215106 2026-02-19T12:49:02.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:49:03.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:49:03.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:03.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 103079215106 2026-02-19T12:49:03.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:49:04.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:49:04.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:04.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215106 -lt 103079215106 2026-02-19T12:49:04.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:04.702 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672964 2026-02-19T12:49:04.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:04.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:49:04.705 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672964 2026-02-19T12:49:04.705 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:04.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672964 2026-02-19T12:49:04.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-19T12:49:04.706 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 42949672964 2026-02-19T12:49:04.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:49:04.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672965 -lt 42949672964 2026-02-19T12:49:04.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:04.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542148 2026-02-19T12:49:04.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:04.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:49:04.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542148 2026-02-19T12:49:04.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:04.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542148 2026-02-19T12:49:04.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-19T12:49:04.871 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542148 2026-02-19T12:49:04.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:49:05.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-19T12:49:05.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:49:05.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:05.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:05.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:49:05.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:49:05.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:49:05.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:49:05.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:49:05.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:49:05.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:49:05.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:49:05.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:49:05.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:49:05.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:05.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:05.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:49:05.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:49:05.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:49:05.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:473: TEST_auto_repair_bluestore_tag: get_pg testpool SOMETHING 2026-02-19T12:49:05.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=testpool 2026-02-19T12:49:05.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:49:05.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map testpool SOMETHING 2026-02-19T12:49:05.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:49:05.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:473: TEST_auto_repair_bluestore_tag: local pgid=1.0 2026-02-19T12:49:05.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:474: TEST_auto_repair_bluestore_tag: get_primary testpool SOMETHING 2026-02-19T12:49:05.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:49:05.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:49:05.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:49:05.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:49:05.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:474: TEST_auto_repair_bluestore_tag: local primary=1 2026-02-19T12:49:05.921 INFO:tasks.workunit.client.0.trial144.stdout:Affected PG 1.0 w/ primary 1 2026-02-19T12:49:05.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:475: TEST_auto_repair_bluestore_tag: echo 'Affected PG ' 1.0 ' w/ primary ' 1 2026-02-19T12:49:05.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:476: TEST_auto_repair_bluestore_tag: get_last_scrub_stamp 1.0 2026-02-19T12:49:05.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:49:05.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:49:05.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:49:05.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:49:06.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:476: TEST_auto_repair_bluestore_tag: local last_scrub_stamp=2026-02-19T12:48:55.285467+0000 2026-02-19T12:49:06.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:477: TEST_auto_repair_bluestore_tag: initiate_and_fetch_state 1 1.0 3.0 2026-02-19T12:49:06.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:382: initiate_and_fetch_state: local the_osd=osd.1 2026-02-19T12:49:06.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:383: initiate_and_fetch_state: local pgid=1.0 2026-02-19T12:49:06.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:384: initiate_and_fetch_state: get_last_scrub_stamp 1.0 2026-02-19T12:49:06.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T12:49:06.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:49:06.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:49:06.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T12:49:06.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:384: initiate_and_fetch_state: local last_scrub=2026-02-19T12:48:55.285467+0000 2026-02-19T12:49:06.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:386: initiate_and_fetch_state: set_config osd 1 osd_scrub_sleep 3.0 2026-02-19T12:49:06.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T12:49:06.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T12:49:06.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=osd_scrub_sleep 2026-02-19T12:49:06.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=3.0 2026-02-19T12:49:06.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T12:49:06.245 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T12:49:06.245 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T12:49:06.245 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T12:49:06.245 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:49:06.245 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:06.246 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:06.246 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T12:49:06.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.1.asok config set osd_scrub_sleep 3.0 2026-02-19T12:49:06.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T12:49:06.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:387: initiate_and_fetch_state: set_config osd 1 osd_scrub_auto_repair true 2026-02-19T12:49:06.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T12:49:06.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T12:49:06.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=osd_scrub_auto_repair 2026-02-19T12:49:06.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T12:49:06.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T12:49:06.307 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T12:49:06.307 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T12:49:06.307 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T12:49:06.307 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:49:06.307 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:06.308 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:06.308 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T12:49:06.308 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.1.asok config set osd_scrub_auto_repair true 2026-02-19T12:49:06.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T12:49:06.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:389: initiate_and_fetch_state: flush_pg_stats 2026-02-19T12:49:06.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:49:06.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:49:06.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:49:06.528 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:49:06.528 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:49:06.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:49:06.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:06.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:49:06.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215108 2026-02-19T12:49:06.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215108 2026-02-19T12:49:06.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-103079215108' 2026-02-19T12:49:06.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:06.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:49:06.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672966 2026-02-19T12:49:06.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672966 2026-02-19T12:49:06.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-103079215108 1-42949672966' 2026-02-19T12:49:06.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:06.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:49:06.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542150 2026-02-19T12:49:06.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542150 2026-02-19T12:49:06.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-103079215108 1-42949672966 2-60129542150' 2026-02-19T12:49:06.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:06.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-103079215108 2026-02-19T12:49:06.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:06.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:49:06.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-103079215108 2026-02-19T12:49:06.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:06.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=103079215108 2026-02-19T12:49:06.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 103079215108' 2026-02-19T12:49:06.751 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 103079215108 2026-02-19T12:49:06.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:06.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215106 -lt 103079215108 2026-02-19T12:49:06.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:49:07.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:49:07.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:08.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215108 -lt 103079215108 2026-02-19T12:49:08.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:08.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672966 2026-02-19T12:49:08.080 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:08.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:49:08.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672966 2026-02-19T12:49:08.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:08.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672966 2026-02-19T12:49:08.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-19T12:49:08.084 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672966 2026-02-19T12:49:08.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:49:08.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672966 -lt 42949672966 2026-02-19T12:49:08.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:08.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542150 2026-02-19T12:49:08.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:08.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:49:08.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542150 2026-02-19T12:49:08.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:08.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542150 2026-02-19T12:49:08.248 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542150 2026-02-19T12:49:08.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-19T12:49:08.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:49:08.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-19T12:49:08.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:390: initiate_and_fetch_state: date --rfc-3339=ns 2026-02-19T12:49:08.408 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19 12:49:08.408719748+00:00 2026-02-19T12:49:08.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:393: initiate_and_fetch_state: get_asok_path osd.1 2026-02-19T12:49:08.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T12:49:08.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T12:49:08.410 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:49:08.410 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:08.410 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:08.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T12:49:08.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:393: initiate_and_fetch_state: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.1.asok schedule-deep-scrub 1.0 2026-02-19T12:49:08.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:396: initiate_and_fetch_state: (( i=0 )) 2026-02-19T12:49:08.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:396: initiate_and_fetch_state: (( i < 80 )) 2026-02-19T12:49:08.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:398: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-19T12:49:08.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:398: initiate_and_fetch_state: jq .state 2026-02-19T12:49:08.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:398: initiate_and_fetch_state: st='"active+clean"' 2026-02-19T12:49:08.553 INFO:tasks.workunit.client.0.trial144.stdout:{"deep":true,"must":false,"stamp":"2026-02-08T08:00:58.469458+0000"}0 ) state now: "active+clean" 2026-02-19T12:49:08.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:399: initiate_and_fetch_state: echo 0 ') state now: ' '"active+clean"' 2026-02-19T12:49:08.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:401: initiate_and_fetch_state: case "$st" in 2026-02-19T12:49:08.554 INFO:tasks.workunit.client.0.trial144.stdout:"active+clean" 2026-02-19T12:49:08.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:406: initiate_and_fetch_state: echo '"active+clean"' 2026-02-19T12:49:08.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:409: initiate_and_fetch_state: '[' 0 == 4 ']' 2026-02-19T12:49:08.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:412: initiate_and_fetch_state: sleep 0.3 2026-02-19T12:49:08.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:396: initiate_and_fetch_state: (( i++ )) 2026-02-19T12:49:08.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:396: initiate_and_fetch_state: (( i < 80 )) 2026-02-19T12:49:08.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:398: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-19T12:49:08.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:398: initiate_and_fetch_state: jq .state 2026-02-19T12:49:08.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:398: initiate_and_fetch_state: st='"active+clean"' 2026-02-19T12:49:08.928 INFO:tasks.workunit.client.0.trial144.stdout:1 ) state now: "active+clean" 2026-02-19T12:49:08.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:399: initiate_and_fetch_state: echo 1 ') state now: ' '"active+clean"' 2026-02-19T12:49:08.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:401: initiate_and_fetch_state: case "$st" in 2026-02-19T12:49:08.929 INFO:tasks.workunit.client.0.trial144.stdout:"active+clean" 2026-02-19T12:49:08.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:406: initiate_and_fetch_state: echo '"active+clean"' 2026-02-19T12:49:08.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:409: initiate_and_fetch_state: '[' 1 == 4 ']' 2026-02-19T12:49:08.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:412: initiate_and_fetch_state: sleep 0.3 2026-02-19T12:49:09.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:396: initiate_and_fetch_state: (( i++ )) 2026-02-19T12:49:09.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:396: initiate_and_fetch_state: (( i < 80 )) 2026-02-19T12:49:09.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:398: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-19T12:49:09.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:398: initiate_and_fetch_state: jq .state 2026-02-19T12:49:09.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:398: initiate_and_fetch_state: st='"active+clean+scrubbing+deep"' 2026-02-19T12:49:09.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:399: initiate_and_fetch_state: echo 2 ') state now: ' '"active+clean+scrubbing+deep"' 2026-02-19T12:49:09.304 INFO:tasks.workunit.client.0.trial144.stdout:2 ) state now: "active+clean+scrubbing+deep" 2026-02-19T12:49:09.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:401: initiate_and_fetch_state: case "$st" in 2026-02-19T12:49:09.305 INFO:tasks.workunit.client.0.trial144.stdout:found scrub 2026-02-19T12:49:09.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:403: initiate_and_fetch_state: echo 'found scrub' 2026-02-19T12:49:09.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:403: initiate_and_fetch_state: return 0 2026-02-19T12:49:09.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:478: TEST_auto_repair_bluestore_tag: r=0 2026-02-19T12:49:09.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:479: TEST_auto_repair_bluestore_tag: echo 'initiate_and_fetch_state ret: ' 0 2026-02-19T12:49:09.306 INFO:tasks.workunit.client.0.trial144.stdout:initiate_and_fetch_state ret: 0 2026-02-19T12:49:09.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:480: TEST_auto_repair_bluestore_tag: set_config osd td/osd-scrub-repair osd_scrub_sleep 0 2026-02-19T12:49:09.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T12:49:09.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=td/osd-scrub-repair 2026-02-19T12:49:09.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=osd_scrub_sleep 2026-02-19T12:49:09.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=0 2026-02-19T12:49:09.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T12:49:09.309 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.td/osd-scrub-repair 2026-02-19T12:49:09.309 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.td/osd-scrub-repair 2026-02-19T12:49:09.309 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.td/osd-scrub-repair ']' 2026-02-19T12:49:09.309 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:49:09.309 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:09.309 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:09.309 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.td/osd-scrub-repair.asok 2026-02-19T12:49:09.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.td/osd-scrub-repair.asok config set osd_scrub_sleep 0 2026-02-19T12:49:09.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test == true 2026-02-19T12:49:09.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh: line 1159: test: ==: unary operator expected 2026-02-19T12:49:09.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:481: TEST_auto_repair_bluestore_tag: '[' 0 -ne 0 ']' 2026-02-19T12:49:09.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:485: TEST_auto_repair_bluestore_tag: wait_end_of_scrub 1 1.0 2026-02-19T12:49:09.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:420: wait_end_of_scrub: local the_osd=osd.1 2026-02-19T12:49:09.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:421: wait_end_of_scrub: local pgid=1.0 2026-02-19T12:49:09.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:423: wait_end_of_scrub: (( i=0 )) 2026-02-19T12:49:09.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:423: wait_end_of_scrub: (( i < 40 )) 2026-02-19T12:49:09.355 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-19T12:49:09.355 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: jq .state 2026-02-19T12:49:09.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: st='"active+clean+scrubbing+deep"' 2026-02-19T12:49:09.424 INFO:tasks.workunit.client.0.trial144.stdout:wait-scrub-end state now: "active+clean+scrubbing+deep" 2026-02-19T12:49:09.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:425: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean+scrubbing+deep"' 2026-02-19T12:49:09.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:426: wait_end_of_scrub: [[ "active+clean+scrubbing+deep" =~ (.*scrubbing.*) ]] 2026-02-19T12:49:09.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:427: wait_end_of_scrub: '[' 0 == 4 ']' 2026-02-19T12:49:09.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:430: wait_end_of_scrub: sleep 0.3 2026-02-19T12:49:09.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:423: wait_end_of_scrub: (( i++ )) 2026-02-19T12:49:09.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:423: wait_end_of_scrub: (( i < 40 )) 2026-02-19T12:49:09.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-19T12:49:09.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: jq .state 2026-02-19T12:49:09.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: st='"active+clean+scrubbing+deep"' 2026-02-19T12:49:09.802 INFO:tasks.workunit.client.0.trial144.stdout:wait-scrub-end state now: "active+clean+scrubbing+deep" 2026-02-19T12:49:09.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:425: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean+scrubbing+deep"' 2026-02-19T12:49:09.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:426: wait_end_of_scrub: [[ "active+clean+scrubbing+deep" =~ (.*scrubbing.*) ]] 2026-02-19T12:49:09.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:427: wait_end_of_scrub: '[' 1 == 4 ']' 2026-02-19T12:49:09.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:430: wait_end_of_scrub: sleep 0.3 2026-02-19T12:49:10.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:423: wait_end_of_scrub: (( i++ )) 2026-02-19T12:49:10.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:423: wait_end_of_scrub: (( i < 40 )) 2026-02-19T12:49:10.104 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-19T12:49:10.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: jq .state 2026-02-19T12:49:10.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: st='"active+clean+scrubbing+deep"' 2026-02-19T12:49:10.177 INFO:tasks.workunit.client.0.trial144.stdout:wait-scrub-end state now: "active+clean+scrubbing+deep" 2026-02-19T12:49:10.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:425: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean+scrubbing+deep"' 2026-02-19T12:49:10.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:426: wait_end_of_scrub: [[ "active+clean+scrubbing+deep" =~ (.*scrubbing.*) ]] 2026-02-19T12:49:10.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:427: wait_end_of_scrub: '[' 2 == 4 ']' 2026-02-19T12:49:10.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:430: wait_end_of_scrub: sleep 0.3 2026-02-19T12:49:10.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:423: wait_end_of_scrub: (( i++ )) 2026-02-19T12:49:10.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:423: wait_end_of_scrub: (( i < 40 )) 2026-02-19T12:49:10.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-19T12:49:10.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: jq .state 2026-02-19T12:49:10.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: st='"active+clean+scrubbing+deep"' 2026-02-19T12:49:10.552 INFO:tasks.workunit.client.0.trial144.stdout:wait-scrub-end state now: "active+clean+scrubbing+deep" 2026-02-19T12:49:10.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:425: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean+scrubbing+deep"' 2026-02-19T12:49:10.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:426: wait_end_of_scrub: [[ "active+clean+scrubbing+deep" =~ (.*scrubbing.*) ]] 2026-02-19T12:49:10.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:427: wait_end_of_scrub: '[' 3 == 4 ']' 2026-02-19T12:49:10.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:430: wait_end_of_scrub: sleep 0.3 2026-02-19T12:49:10.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:423: wait_end_of_scrub: (( i++ )) 2026-02-19T12:49:10.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:423: wait_end_of_scrub: (( i < 40 )) 2026-02-19T12:49:10.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-19T12:49:10.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: jq .state 2026-02-19T12:49:10.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: st='"active+clean+scrubbing+deep"' 2026-02-19T12:49:10.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:425: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean+scrubbing+deep"' 2026-02-19T12:49:10.926 INFO:tasks.workunit.client.0.trial144.stdout:wait-scrub-end state now: "active+clean+scrubbing+deep" 2026-02-19T12:49:10.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:426: wait_end_of_scrub: [[ "active+clean+scrubbing+deep" =~ (.*scrubbing.*) ]] 2026-02-19T12:49:10.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:427: wait_end_of_scrub: '[' 4 == 4 ']' 2026-02-19T12:49:10.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:428: wait_end_of_scrub: flush_pg_stats 2026-02-19T12:49:10.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:49:10.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:49:11.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:49:11.085 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:49:11.086 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:49:11.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:49:11.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:11.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:49:11.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215109 2026-02-19T12:49:11.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215109 2026-02-19T12:49:11.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-103079215109' 2026-02-19T12:49:11.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:11.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:49:11.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672968 2026-02-19T12:49:11.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672968 2026-02-19T12:49:11.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-103079215109 1-42949672968' 2026-02-19T12:49:11.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:11.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:49:11.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542152 2026-02-19T12:49:11.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542152 2026-02-19T12:49:11.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-103079215109 1-42949672968 2-60129542152' 2026-02-19T12:49:11.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:11.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-103079215109 2026-02-19T12:49:11.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:11.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:49:11.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-103079215109 2026-02-19T12:49:11.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:11.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=103079215109 2026-02-19T12:49:11.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 103079215109' 2026-02-19T12:49:11.306 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 103079215109 2026-02-19T12:49:11.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:11.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215108 -lt 103079215109 2026-02-19T12:49:11.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:49:12.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:49:12.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215110 -lt 103079215109 2026-02-19T12:49:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:12.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672968 2026-02-19T12:49:12.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:12.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:49:12.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672968 2026-02-19T12:49:12.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:12.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672968 2026-02-19T12:49:12.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2026-02-19T12:49:12.634 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672968 2026-02-19T12:49:12.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:49:12.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672968 -lt 42949672968 2026-02-19T12:49:12.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:12.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542152 2026-02-19T12:49:12.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:12.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:49:12.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542152 2026-02-19T12:49:12.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:12.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542152 2026-02-19T12:49:12.805 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542152 2026-02-19T12:49:12.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542152' 2026-02-19T12:49:12.806 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:49:12.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542152 -lt 60129542152 2026-02-19T12:49:12.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:430: wait_end_of_scrub: sleep 0.3 2026-02-19T12:49:13.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:423: wait_end_of_scrub: (( i++ )) 2026-02-19T12:49:13.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:423: wait_end_of_scrub: (( i < 40 )) 2026-02-19T12:49:13.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-19T12:49:13.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: jq .state 2026-02-19T12:49:13.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:424: wait_end_of_scrub: st='"active+clean"' 2026-02-19T12:49:13.353 INFO:tasks.workunit.client.0.trial144.stdout:wait-scrub-end state now: "active+clean" 2026-02-19T12:49:13.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:425: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean"' 2026-02-19T12:49:13.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:426: wait_end_of_scrub: [[ "active+clean" =~ (.*scrubbing.*) ]] 2026-02-19T12:49:13.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:426: wait_end_of_scrub: break 2026-02-19T12:49:13.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:433: wait_end_of_scrub: [[ "active+clean" =~ (.*scrubbing.*) ]] 2026-02-19T12:49:13.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:438: wait_end_of_scrub: return 0 2026-02-19T12:49:13.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:486: TEST_auto_repair_bluestore_tag: ceph pg dump pgs 2026-02-19T12:49:13.497 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T12:49:13.497 INFO:tasks.workunit.client.0.trial144.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean+scrubbing+deep 2026-02-19T12:49:09.191197+0000 20'1 25:52 [1,0] 1 [1,0] 1 0'0 2026-02-19T12:48:55.285467+0000 0'0 2026-02-08T08:00:58.469458+0000 0 0 deep scrubbing for 0s 0 0 2026-02-19T12:49:13.497 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T12:49:13.497 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T12:49:13.497 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T12:49:13.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:490: TEST_auto_repair_bluestore_tag: get_not_primary testpool SOMETHING 2026-02-19T12:49:13.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=testpool 2026-02-19T12:49:13.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:49:13.507 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary testpool SOMETHING 2026-02-19T12:49:13.507 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:49:13.507 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:49:13.507 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:49:13.507 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:49:13.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:49:13.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:49:13.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:49:13.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:490: TEST_auto_repair_bluestore_tag: objectstore_tool td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-19T12:49:13.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:49:13.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:49:13.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:49:13.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:49:13.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-19T12:49:13.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:49:13.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:49:13.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:49:13.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:49:13.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:49:13.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:49:13.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:49:13.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:49:13.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:49:13.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:49:14.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:49:14.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-19T12:49:14.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:49:14.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:49:14.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:49:14.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:49:14.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:49:14.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING list-attrs 2026-02-19T12:49:14.197 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:49:14.197 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:49:14.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-19T12:49:14.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:49:14.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:49:14.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:49:14.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:49:14.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:49:14.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:49:14.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:49:14.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:49:14.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:49:14.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:49:14.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:49:14.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:49:14.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:49:14.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:49:14.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:49:14.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:49:14.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:49:14.295 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:49:14.295 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:14.296 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:14.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:14.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:14.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:49:14.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:49:14.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:49:14.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:49:14.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:49:14.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:49:14.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:49:14.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:49:14.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-19T12:49:14.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:49:14.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:49:14.298 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:49:14.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-19T12:49:14.298 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:49:14.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:49:14.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:49:14.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:49:14.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:49:14.315 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:14.314+0000 7fd7604f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:14.317 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:14.314+0000 7fd7604f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:14.318 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:14.314+0000 7fd7604f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:14.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:49:14.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:49:14.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:49:14.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:49:14.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:49:14.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:14.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:49:14.459 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:49:14.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:14.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:49:14.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:49:14.713 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:14.710+0000 7fd7604f1a40 -1 Falling back to public interface 2026-02-19T12:49:15.294 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:15.290+0000 7fd7604f1a40 -1 osd.0 25 log_to_monitors true 2026-02-19T12:49:15.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:49:15.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:15.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:49:15.618 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:49:15.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:15.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:49:15.779 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 29 up_thru 0 down_at 26 last_clean_interval [24,25) [v2:127.0.0.1:6802/1435327350,v1:127.0.0.1:6803/1435327350] [v2:127.0.0.1:6804/1435327350,v1:127.0.0.1:6805/1435327350] exists,up ac4b7d16-5de1-420d-9846-ba20aca6d6d7 2026-02-19T12:49:15.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:49:15.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:49:15.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:49:15.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:49:15.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:49:15.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:49:15.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:49:15.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:49:15.781 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:49:15.781 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:49:15.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:49:15.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:49:15.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:49:15.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:49:15.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:49:15.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:49:15.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:49:15.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:49:15.881 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:49:16.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:49:16.036 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:49:16.036 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:49:16.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:49:16.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:16.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:49:16.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051586 2026-02-19T12:49:16.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051586 2026-02-19T12:49:16.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-124554051586' 2026-02-19T12:49:16.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:16.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:49:16.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672970 2026-02-19T12:49:16.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672970 2026-02-19T12:49:16.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-124554051586 1-42949672970' 2026-02-19T12:49:16.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:16.182 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:49:16.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542154 2026-02-19T12:49:16.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542154 2026-02-19T12:49:16.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-124554051586 1-42949672970 2-60129542154' 2026-02-19T12:49:16.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:16.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-124554051586 2026-02-19T12:49:16.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:16.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:49:16.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-124554051586 2026-02-19T12:49:16.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:16.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051586 2026-02-19T12:49:16.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 124554051586' 2026-02-19T12:49:16.260 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 124554051586 2026-02-19T12:49:16.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:16.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 124554051586 2026-02-19T12:49:16.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:49:17.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:49:17.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:17.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 124554051586 2026-02-19T12:49:17.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:49:18.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:49:18.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:18.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051586 -lt 124554051586 2026-02-19T12:49:18.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:18.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672970 2026-02-19T12:49:18.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:18.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:49:18.744 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672970 2026-02-19T12:49:18.744 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:18.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672970 2026-02-19T12:49:18.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-19T12:49:18.746 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 42949672970 2026-02-19T12:49:18.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:49:18.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672970 -lt 42949672970 2026-02-19T12:49:18.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:18.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542154 2026-02-19T12:49:18.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:18.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:49:18.908 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542154 2026-02-19T12:49:18.908 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:18.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542154 2026-02-19T12:49:18.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542154' 2026-02-19T12:49:18.910 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542154 2026-02-19T12:49:18.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:49:19.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542154 -lt 60129542154 2026-02-19T12:49:19.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:49:19.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:19.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:19.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:49:19.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:49:19.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:49:19.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:49:19.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:49:19.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:49:19.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:49:19.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:49:19.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T12:49:19.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:49:19.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:19.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:19.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T12:49:19.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-19T12:49:19.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T12:49:19.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-19T12:49:19.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:49:19.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T12:49:19.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:49:19.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:49:19.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:49:19.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:49:19.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:49:19.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:49:19.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:49:19.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:49:19.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T12:49:19.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:49:19.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:19.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:20.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T12:49:20.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-19T12:49:20.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:49:20.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:49:20.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:49:20.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:49:20.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:49:20.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:49:20.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:49:20.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:49:20.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:49:20.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T12:49:20.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:49:20.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T12:49:20.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:49:20.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:49:20.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:49:20.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:49:20.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:49:20.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:49:20.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:49:20.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:49:20.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:49:20.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:49:20.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:20.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:20.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:49:20.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:49:20.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:49:20.819 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:491: TEST_auto_repair_bluestore_tag: get_not_primary testpool SOMETHING 2026-02-19T12:49:20.819 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=testpool 2026-02-19T12:49:20.819 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:49:20.819 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary testpool SOMETHING 2026-02-19T12:49:20.820 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T12:49:20.820 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:49:20.820 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:49:20.820 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:49:20.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:49:20.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-19T12:49:20.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:49:21.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:491: TEST_auto_repair_bluestore_tag: objectstore_tool td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-19T12:49:21.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:49:21.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:49:21.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:49:21.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:49:21.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-19T12:49:21.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:49:21.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:49:21.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:49:21.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:49:21.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:49:21.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:49:21.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:49:21.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:49:21.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:49:21.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:49:21.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:49:21.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-19T12:49:21.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:49:21.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:49:21.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:49:21.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:49:21.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:49:21.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-19T12:49:21.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-19T12:49:21.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:49:21.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:49:21.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:49:21.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:49:21.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:49:21.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:49:21.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:49:21.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:49:21.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:49:21.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:49:21.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:49:21.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:49:21.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:49:21.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:49:21.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:49:21.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:49:21.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:49:21.627 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:49:21.627 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:21.628 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:21.628 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:21.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:21.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:49:21.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:49:21.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:49:21.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:49:21.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:49:21.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:49:21.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:49:21.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:49:21.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-19T12:49:21.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:49:21.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:49:21.630 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:49:21.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-19T12:49:21.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:49:21.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:49:21.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:49:21.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:49:21.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:49:21.653 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:21.649+0000 7fba12e3ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:21.654 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:21.649+0000 7fba12e3ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:21.655 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:21.649+0000 7fba12e3ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:21.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:49:21.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:49:21.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:49:21.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:49:21.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:49:21.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:21.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:49:21.786 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:49:21.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:21.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:49:21.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:49:22.025 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:22.021+0000 7fba12e3ca40 -1 Falling back to public interface 2026-02-19T12:49:22.621 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:22.617+0000 7fba12e3ca40 -1 osd.0 30 log_to_monitors true 2026-02-19T12:49:22.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:49:22.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:22.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:49:22.949 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:49:22.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:22.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:49:23.107 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 34 up_thru 0 down_at 31 last_clean_interval [29,30) [v2:127.0.0.1:6802/2748526089,v1:127.0.0.1:6803/2748526089] [v2:127.0.0.1:6804/2748526089,v1:127.0.0.1:6805/2748526089] exists,up ac4b7d16-5de1-420d-9846-ba20aca6d6d7 2026-02-19T12:49:23.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:49:23.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:49:23.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:49:23.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:49:23.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:49:23.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:49:23.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:49:23.109 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:49:23.109 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:49:23.110 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:49:23.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:49:23.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:49:23.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:49:23.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:49:23.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:49:23.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:49:23.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:49:23.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:49:23.216 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:49:23.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:49:23.375 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:49:23.375 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:49:23.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:49:23.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:23.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:49:23.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888066 2026-02-19T12:49:23.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888066 2026-02-19T12:49:23.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-146028888066' 2026-02-19T12:49:23.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:23.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:49:23.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672973 2026-02-19T12:49:23.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672973 2026-02-19T12:49:23.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-146028888066 1-42949672973' 2026-02-19T12:49:23.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:23.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:49:23.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542156 2026-02-19T12:49:23.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542156 2026-02-19T12:49:23.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-146028888066 1-42949672973 2-60129542156' 2026-02-19T12:49:23.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:23.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-146028888066 2026-02-19T12:49:23.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:23.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:49:23.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-146028888066 2026-02-19T12:49:23.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:23.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888066 2026-02-19T12:49:23.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 146028888066' 2026-02-19T12:49:23.598 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 146028888066 2026-02-19T12:49:23.598 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:23.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051587 -lt 146028888066 2026-02-19T12:49:23.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:49:24.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:49:24.763 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:24.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888066 -lt 146028888066 2026-02-19T12:49:24.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:24.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672973 2026-02-19T12:49:24.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:24.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:49:24.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672973 2026-02-19T12:49:24.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:24.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672973 2026-02-19T12:49:24.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672973' 2026-02-19T12:49:24.928 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 42949672973 2026-02-19T12:49:24.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:49:25.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672973 -lt 42949672973 2026-02-19T12:49:25.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:25.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542156 2026-02-19T12:49:25.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:25.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:49:25.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542156 2026-02-19T12:49:25.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:25.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542156 2026-02-19T12:49:25.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542156' 2026-02-19T12:49:25.094 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542156 2026-02-19T12:49:25.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:49:25.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542156 -lt 60129542156 2026-02-19T12:49:25.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:49:25.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:25.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:25.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T12:49:25.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:49:25.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:49:25.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:49:25.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:49:25.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:49:25.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:49:25.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:49:25.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T12:49:25.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:49:25.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:25.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:25.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T12:49:25.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:49:25.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:49:25.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:492: TEST_auto_repair_bluestore_tag: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:49:25.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:493: TEST_auto_repair_bluestore_tag: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-19T12:49:25.820 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:49:12.190+0000 7fa63e8ca640 10 osd.1 pg_epoch: 25 pg[1.0( v 20'1 (0'0,20'1] local-lis/les=24/25 n=1 ec=15/15 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=0 lpr=24 crt=20'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep(periodic-regular,auto-repair)] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-19T12:49:25.820 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:49:12.190+0000 7fa63e8ca640 10 osd.1 pg_epoch: 25 pg[1.0( v 20'1 (0'0,20'1] local-lis/les=24/25 n=1 ec=15/15 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=0 lpr=24 crt=20'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep(periodic-regular,auto-repair)] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-19T12:49:25.820 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:49:12.190+0000 7fa63e8ca640 15 osd.1 pg_epoch: 25 pg[1.0( v 20'1 (0'0,20'1] local-lis/les=24/25 n=1 ec=15/15 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=0 lpr=24 crt=20'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair(periodic-regular,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish: 1 errors. 1 errors fixed 2026-02-19T12:49:25.821 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:49:12.190+0000 7fa63e8ca640 20 osd.1 pg_epoch: 25 pg[1.0( v 20'1 (0'0,20'1] local-lis/les=24/25 n=1 ec=15/15 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=0 lpr=24 crt=20'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair(periodic-regular,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish All may be fixed 2026-02-19T12:49:25.821 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T12:49:12.190+0000 7fa63e8ca640 19 osd.1 pg_epoch: 25 pg[1.0( v 20'1 (0'0,20'1] local-lis/les=24/25 n=1 ec=15/15 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=0 lpr=24 crt=20'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair(periodic-regular,auto-repair) mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-19T12:49:25.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T12:49:25.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:49:25.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:49:25.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:49:25.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:49:25.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:49:25.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:49:25.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:49:25.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:49:25.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:49:25.932 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:49:25.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:49:25.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:49:25.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:49:25.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:49:25.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:49:25.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:49:25.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:49:25.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:49:25.937 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:49:25.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:49:25.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:49:25.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:49:25.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:49:25.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:25.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:25.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:49:25.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:49:25.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:49:25.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T12:49:25.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T12:49:25.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T12:49:25.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T12:49:25.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:49:25.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:49:25.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:49:25.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:49:25.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:49:25.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:49:25.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:49:25.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:49:25.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:49:25.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:49:25.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:49:25.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:49:25.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:49:25.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:49:25.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:49:25.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:49:25.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:49:25.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:49:25.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:49:25.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:49:25.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:49:25.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:49:25.962 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:49:25.962 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:25.962 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:25.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:49:25.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:49:25.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:49:25.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T12:49:25.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:49:25.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:25.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:25.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T12:49:25.967 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:49:25.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T12:49:25.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:49:25.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T12:49:25.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_erasure_coded_appends td/osd-scrub-repair 2026-02-19T12:49:25.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:369: TEST_auto_repair_erasure_coded_appends: auto_repair_erasure_coded td/osd-scrub-repair false 2026-02-19T12:49:25.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:328: auto_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-19T12:49:25.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:329: auto_repair_erasure_coded: local allow_overwrites=false 2026-02-19T12:49:25.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:330: auto_repair_erasure_coded: local poolname=ecpool 2026-02-19T12:49:25.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:333: auto_repair_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-19T12:49:25.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T12:49:25.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T12:49:25.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T12:49:25.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T12:49:25.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T12:49:25.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T12:49:25.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T12:49:25.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:49:25.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:49:25.991 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:49:25.991 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:25.991 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:25.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:25.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T12:49:26.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T12:49:26.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T12:49:26.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:49:26.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:49:26.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T12:49:26.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T12:49:26.022 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:49:26.022 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:49:26.022 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:49:26.022 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:49:26.022 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:26.022 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:26.022 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:49:26.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:49:26.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T12:49:26.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T12:49:26.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:49:26.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:49:26.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T12:49:26.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T12:49:26.078 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:49:26.078 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:49:26.078 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:49:26.079 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:49:26.079 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:26.079 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:26.079 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:49:26.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:49:26.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T12:49:26.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:334: auto_repair_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-19T12:49:26.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T12:49:26.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T12:49:26.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T12:49:26.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T12:49:26.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T12:49:26.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T12:49:26.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T12:49:26.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:49:26.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:49:26.230 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:49:26.230 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:26.230 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:26.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:26.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:49:26.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:49:26.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: auto_repair_erasure_coded: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:49:26.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:340: auto_repair_erasure_coded: seq 0 2 2026-02-19T12:49:26.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:340: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-19T12:49:26.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:49:26.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:49:26.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:49:26.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T12:49:26.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:49:26.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:49:26.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:49:26.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:49:26.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:49:26.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:49:26.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:49:26.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:49:26.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:49:26.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:49:26.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:49:26.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:49:26.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:49:26.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:49:26.254 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:49:26.254 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:26.254 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:26.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:26.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:26.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:49:26.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:49:26.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:49:26.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:49:26.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:49:26.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:49:26.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:49:26.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:49:26.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:49:26.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:49:26.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:49:26.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:49:26.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:49:26.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ef74d0b1-930b-4ad0-ad9b-a6fcae32ee67 2026-02-19T12:49:26.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ef74d0b1-930b-4ad0-ad9b-a6fcae32ee67' 2026-02-19T12:49:26.257 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 ef74d0b1-930b-4ad0-ad9b-a6fcae32ee67 2026-02-19T12:49:26.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:49:26.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDWBpdpgQTNDxAAwRu3UWJ8XKip/+hQLEsjwA== 2026-02-19T12:49:26.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDWBpdpgQTNDxAAwRu3UWJ8XKip/+hQLEsjwA=="}' 2026-02-19T12:49:26.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ef74d0b1-930b-4ad0-ad9b-a6fcae32ee67 -i td/osd-scrub-repair/0/new.json 2026-02-19T12:49:26.354 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:49:26.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T12:49:26.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDWBpdpgQTNDxAAwRu3UWJ8XKip/+hQLEsjwA== --osd-uuid ef74d0b1-930b-4ad0-ad9b-a6fcae32ee67 2026-02-19T12:49:26.382 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:26.377+0000 7fa01c74ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:26.383 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:26.381+0000 7fa01c74ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:26.384 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:26.381+0000 7fa01c74ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:26.385 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:26.381+0000 7fa01c74ea40 -1 bdev(0x55f1296eac00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:49:26.385 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:26.381+0000 7fa01c74ea40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T12:49:27.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T12:49:27.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:49:27.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T12:49:27.156 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:49:27.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:49:27.363 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:49:27.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T12:49:27.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:49:27.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:49:27.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:49:27.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:49:27.391 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:27.389+0000 7f0a59f5ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:27.393 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:27.389+0000 7f0a59f5ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:27.394 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:27.389+0000 7f0a59f5ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:27.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T12:49:27.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:49:27.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:49:27.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:49:27.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:49:27.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:27.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:49:27.523 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:49:27.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:27.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:49:27.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:49:27.785 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:27.781+0000 7f0a59f5ba40 -1 Falling back to public interface 2026-02-19T12:49:28.405 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:28.401+0000 7f0a59f5ba40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:49:28.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:49:28.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:28.681 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:49:28.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:49:28.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:28.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:49:28.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:49:29.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:49:29.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:29.840 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:49:29.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:49:29.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:49:29.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:29.997 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/104691421,v1:127.0.0.1:6803/104691421] [v2:127.0.0.1:6804/104691421,v1:127.0.0.1:6805/104691421] exists,up ef74d0b1-930b-4ad0-ad9b-a6fcae32ee67 2026-02-19T12:49:30.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:49:30.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:49:30.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:49:30.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:340: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-19T12:49:30.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:49:30.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:49:30.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:49:30.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T12:49:30.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:49:30.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:49:30.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:49:30.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:49:30.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:49:30.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:49:30.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:49:30.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:49:30.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:49:30.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:49:30.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:49:30.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:49:30.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:49:30.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:49:30.003 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:49:30.003 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:30.003 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:30.003 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:30.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:30.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:49:30.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:49:30.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:49:30.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:49:30.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:49:30.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:49:30.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:49:30.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:49:30.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:49:30.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:49:30.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:49:30.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:49:30.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:49:30.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cf17b739-254d-4d74-9793-8d4e37d2fdda 2026-02-19T12:49:30.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 cf17b739-254d-4d74-9793-8d4e37d2fdda' 2026-02-19T12:49:30.005 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 cf17b739-254d-4d74-9793-8d4e37d2fdda 2026-02-19T12:49:30.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:49:30.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDaBpdpDKLNABAAGQfP6TS7zm9A4CkzY0n3Vw== 2026-02-19T12:49:30.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDaBpdpDKLNABAAGQfP6TS7zm9A4CkzY0n3Vw=="}' 2026-02-19T12:49:30.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cf17b739-254d-4d74-9793-8d4e37d2fdda -i td/osd-scrub-repair/1/new.json 2026-02-19T12:49:30.177 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:49:30.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T12:49:30.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDaBpdpDKLNABAAGQfP6TS7zm9A4CkzY0n3Vw== --osd-uuid cf17b739-254d-4d74-9793-8d4e37d2fdda 2026-02-19T12:49:30.216 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:30.213+0000 7fb7b4b0fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:30.217 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:30.213+0000 7fb7b4b0fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:30.218 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:30.213+0000 7fb7b4b0fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:30.219 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:30.213+0000 7fb7b4b0fa40 -1 bdev(0x55560e08ac00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:49:30.219 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:30.213+0000 7fb7b4b0fa40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T12:49:30.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T12:49:30.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:49:30.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T12:49:30.863 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:49:30.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:49:31.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T12:49:31.067 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:49:31.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:49:31.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:49:31.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:49:31.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:49:31.094 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:31.089+0000 7fb1af79ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:31.096 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:31.093+0000 7fb1af79ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:31.097 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:31.093+0000 7fb1af79ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:31.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T12:49:31.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:49:31.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:49:31.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:49:31.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:49:31.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:31.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:49:31.226 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:49:31.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:31.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:49:31.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:49:31.465 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:31.461+0000 7fb1af79ba40 -1 Falling back to public interface 2026-02-19T12:49:32.098 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:32.093+0000 7fb1af79ba40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:49:32.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:49:32.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:32.385 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:49:32.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:49:32.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:32.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:49:32.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:49:33.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:49:33.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:33.551 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:49:33.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:49:33.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:33.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:49:33.709 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1180814300,v1:127.0.0.1:6811/1180814300] [v2:127.0.0.1:6812/1180814300,v1:127.0.0.1:6813/1180814300] exists,up cf17b739-254d-4d74-9793-8d4e37d2fdda 2026-02-19T12:49:33.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:49:33.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:49:33.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:49:33.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:340: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-19T12:49:33.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:49:33.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:49:33.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:49:33.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T12:49:33.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:49:33.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:49:33.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:49:33.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:49:33.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:49:33.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:49:33.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:49:33.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:49:33.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:49:33.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:49:33.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:49:33.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:49:33.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:49:33.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:49:33.716 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:49:33.716 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:33.716 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:33.716 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:33.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:33.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:49:33.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:49:33.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:49:33.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:49:33.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:49:33.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:49:33.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:49:33.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:49:33.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:49:33.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:49:33.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:49:33.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:49:33.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:49:33.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2558dead-f168-4289-b3e6-2327eb26f45d 2026-02-19T12:49:33.718 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 2558dead-f168-4289-b3e6-2327eb26f45d 2026-02-19T12:49:33.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 2558dead-f168-4289-b3e6-2327eb26f45d' 2026-02-19T12:49:33.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:49:33.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDdBpdpxedFKxAAjWL+yGzt2H8Yc/0pxORskw== 2026-02-19T12:49:33.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDdBpdpxedFKxAAjWL+yGzt2H8Yc/0pxORskw=="}' 2026-02-19T12:49:33.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2558dead-f168-4289-b3e6-2327eb26f45d -i td/osd-scrub-repair/2/new.json 2026-02-19T12:49:33.888 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:49:33.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T12:49:33.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDdBpdpxedFKxAAjWL+yGzt2H8Yc/0pxORskw== --osd-uuid 2558dead-f168-4289-b3e6-2327eb26f45d 2026-02-19T12:49:33.925 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:33.921+0000 7f76e1194a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:33.927 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:33.921+0000 7f76e1194a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:33.928 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:33.925+0000 7f76e1194a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:33.929 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:33.925+0000 7f76e1194a40 -1 bdev(0x560ade9f8c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T12:49:33.929 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:33.925+0000 7f76e1194a40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T12:49:34.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T12:49:34.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:49:34.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T12:49:34.569 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T12:49:34.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:49:34.769 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T12:49:34.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T12:49:34.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:49:34.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:49:34.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:49:34.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:49:34.798 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:34.793+0000 7fe43aa7da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:34.800 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:34.797+0000 7fe43aa7da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:34.801 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:34.797+0000 7fe43aa7da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:34.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T12:49:34.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:49:34.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:49:34.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:49:34.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:49:34.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:34.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:49:34.929 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:49:34.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:34.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:49:35.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:49:35.209 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:35.205+0000 7fe43aa7da40 -1 Falling back to public interface 2026-02-19T12:49:35.816 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:35.813+0000 7fe43aa7da40 -1 osd.2 0 log_to_monitors true 2026-02-19T12:49:36.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:49:36.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:36.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:49:36.088 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:49:36.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:36.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:49:36.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:49:37.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:49:37.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:37.249 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:49:37.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:49:37.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:37.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:49:37.407 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/756241956,v1:127.0.0.1:6819/756241956] [v2:127.0.0.1:6820/756241956,v1:127.0.0.1:6821/756241956] exists,up 2558dead-f168-4289-b3e6-2327eb26f45d 2026-02-19T12:49:37.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:49:37.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:49:37.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:49:37.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:343: auto_repair_erasure_coded: create_rbd_pool 2026-02-19T12:49:37.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T12:49:37.558 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T12:49:37.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T12:49:37.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T12:49:37.775 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T12:49:37.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:49:38.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T12:49:39.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:344: auto_repair_erasure_coded: wait_for_clean 2026-02-19T12:49:39.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:49:39.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:49:39.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:49:39.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:49:39.082 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:49:39.082 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:49:39.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:49:39.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:49:39.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:49:39.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:49:39.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:49:39.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:49:39.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:49:39.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:49:39.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:49:39.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:49:39.348 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:49:39.349 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:49:39.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:49:39.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:39.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:49:39.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T12:49:39.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T12:49:39.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T12:49:39.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:39.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:49:39.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-19T12:49:39.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-19T12:49:39.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705667' 2026-02-19T12:49:39.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:39.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:49:39.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542146 2026-02-19T12:49:39.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542146 2026-02-19T12:49:39.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705667 2-60129542146' 2026-02-19T12:49:39.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:39.573 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T12:49:39.573 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:39.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:49:39.574 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T12:49:39.574 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:39.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T12:49:39.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T12:49:39.576 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T12:49:39.577 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:39.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T12:49:39.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:49:40.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:49:40.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:40.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836483 2026-02-19T12:49:40.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:40.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-19T12:49:40.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:40.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:49:40.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705667 2026-02-19T12:49:40.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:40.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705667 2026-02-19T12:49:40.898 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705667 2026-02-19T12:49:40.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-19T12:49:40.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:49:41.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705667 -lt 38654705667 2026-02-19T12:49:41.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:41.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542146 2026-02-19T12:49:41.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:41.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:49:41.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542146 2026-02-19T12:49:41.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:41.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542146 2026-02-19T12:49:41.069 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542146 2026-02-19T12:49:41.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-19T12:49:41.070 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:49:41.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542146 -lt 60129542146 2026-02-19T12:49:41.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:49:41.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:41.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:41.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:49:41.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:49:41.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:49:41.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:49:41.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:49:41.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:49:41.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:49:41.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:49:41.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:49:41.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:49:41.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:41.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:41.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:49:41.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:49:41.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:49:41.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:347: auto_repair_erasure_coded: create_ec_pool ecpool false k=2 m=1 2026-02-19T12:49:41.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: create_ec_pool: local pool_name=ecpool 2026-02-19T12:49:41.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: create_ec_pool: shift 2026-02-19T12:49:41.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: create_ec_pool: local allow_overwrites=false 2026-02-19T12:49:41.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2525: create_ec_pool: shift 2026-02-19T12:49:41.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-19T12:49:41.974 INFO:tasks.workunit.client.0.trial144.stderr:load: isa 2026-02-19T12:49:41.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-19T12:49:41.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-19T12:49:42.279 INFO:tasks.workunit.client.0.trial144.stderr:pool 'ecpool' created 2026-02-19T12:49:42.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:49:43.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: create_ec_pool: '[' false = true ']' 2026-02-19T12:49:43.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2535: create_ec_pool: wait_for_clean 2026-02-19T12:49:43.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:49:43.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:49:43.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:49:43.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:49:43.297 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:49:43.297 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:49:43.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:49:43.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:49:43.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:49:43.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:49:43.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:49:43.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:49:43.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:49:43.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:49:43.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:49:43.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:49:43.572 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:49:43.572 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:49:43.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:49:43.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:43.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:49:43.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-19T12:49:43.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-19T12:49:43.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485' 2026-02-19T12:49:43.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:43.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:49:43.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705669 2026-02-19T12:49:43.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705669 2026-02-19T12:49:43.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-38654705669' 2026-02-19T12:49:43.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:43.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:49:43.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542148 2026-02-19T12:49:43.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542148 2026-02-19T12:49:43.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-38654705669 2-60129542148' 2026-02-19T12:49:43.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:43.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-19T12:49:43.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:43.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:49:43.796 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836485 2026-02-19T12:49:43.796 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:43.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836485 2026-02-19T12:49:43.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-19T12:49:43.798 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836485 2026-02-19T12:49:43.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:43.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-19T12:49:43.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:49:44.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:49:44.962 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:45.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-19T12:49:45.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:45.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705669 2026-02-19T12:49:45.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:45.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:49:45.126 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705669 2026-02-19T12:49:45.126 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:45.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705669 2026-02-19T12:49:45.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-19T12:49:45.128 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705669 2026-02-19T12:49:45.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:49:45.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-19T12:49:45.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:45.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542148 2026-02-19T12:49:45.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:45.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:49:45.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542148 2026-02-19T12:49:45.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:45.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542148 2026-02-19T12:49:45.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-19T12:49:45.290 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542148 2026-02-19T12:49:45.290 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:49:45.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-19T12:49:45.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:49:45.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:45.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:45.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:49:45.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:49:45.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:49:45.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:49:45.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:49:45.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:49:45.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:49:45.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:49:45.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:49:45.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:49:45.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:45.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:46.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:49:46.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:49:46.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:49:46.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2536: create_ec_pool: return 0 2026-02-19T12:49:46.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:350: auto_repair_erasure_coded: local payload=ABCDEF 2026-02-19T12:49:46.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:351: auto_repair_erasure_coded: echo ABCDEF 2026-02-19T12:49:46.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: auto_repair_erasure_coded: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-19T12:49:46.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:356: auto_repair_erasure_coded: get_not_primary ecpool SOMETHING 2026-02-19T12:49:46.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=ecpool 2026-02-19T12:49:46.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:49:46.040 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary ecpool SOMETHING 2026-02-19T12:49:46.040 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=ecpool 2026-02-19T12:49:46.040 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:49:46.041 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:49:46.041 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:49:46.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=2 2026-02-19T12:49:46.205 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:49:46.205 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 2)) | .[0]' 2026-02-19T12:49:46.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:356: auto_repair_erasure_coded: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:49:46.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:49:46.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:49:46.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:49:46.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:49:46.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:49:46.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:49:46.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:49:46.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:49:46.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:49:46.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:49:46.365 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:49:46.365 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:49:46.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:49:46.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:49:46.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:49:46.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:49:46.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:49:46.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:49:46.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:49:46.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:49:46.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:49:46.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:49:46.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-19T12:49:46.871 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-19T12:49:46.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:49:46.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:49:46.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:49:46.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:49:46.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:49:46.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:49:46.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:49:46.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:49:46.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:49:46.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:49:46.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:49:46.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:49:46.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:49:46.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:49:46.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:49:46.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:49:46.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:49:46.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:49:46.972 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:49:46.972 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:46.972 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:46.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:46.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:46.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:49:46.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:49:46.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:49:46.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:49:46.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:49:46.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:49:46.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:49:46.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:49:46.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:49:46.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:49:46.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:49:46.973 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:49:46.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:49:46.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:49:46.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:49:46.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:49:46.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:49:46.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:49:46.992 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:46.989+0000 7fd0c92d9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:46.993 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:46.989+0000 7fd0c92d9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:46.994 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:46.989+0000 7fd0c92d9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:47.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:49:47.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:49:47.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:49:47.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:49:47.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:49:47.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:47.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:49:47.132 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:49:47.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:47.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:49:47.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:49:47.365 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:47.361+0000 7fd0c92d9a40 -1 Falling back to public interface 2026-02-19T12:49:48.010 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:48.005+0000 7fd0c92d9a40 -1 osd.1 27 log_to_monitors true 2026-02-19T12:49:48.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:49:48.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:48.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:49:48.289 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:49:48.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:48.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:49:48.448 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 31 up_thru 31 down_at 28 last_clean_interval [9,27) [v2:127.0.0.1:6810/3887015039,v1:127.0.0.1:6811/3887015039] [v2:127.0.0.1:6812/3887015039,v1:127.0.0.1:6813/3887015039] exists,up cf17b739-254d-4d74-9793-8d4e37d2fdda 2026-02-19T12:49:48.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:49:48.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:49:48.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:49:48.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:49:48.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:49:48.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:49:48.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:49:48.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:49:48.449 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:49:48.449 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:49:48.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:49:48.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:49:48.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:49:48.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:49:48.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:49:48.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:49:48.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:49:48.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:49:48.557 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:49:48.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:49:48.712 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:49:48.712 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:49:48.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:49:48.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:48.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:49:48.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-19T12:49:48.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-19T12:49:48.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-19T12:49:48.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:48.786 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:49:48.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=133143986178 2026-02-19T12:49:48.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 133143986178 2026-02-19T12:49:48.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-133143986178' 2026-02-19T12:49:48.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:48.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:49:48.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542150 2026-02-19T12:49:48.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542150 2026-02-19T12:49:48.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-133143986178 2-60129542150' 2026-02-19T12:49:48.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:48.931 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-19T12:49:48.931 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:48.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:49:48.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-19T12:49:48.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:48.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-19T12:49:48.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-19T12:49:48.934 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836487 2026-02-19T12:49:48.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:49.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T12:49:49.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:49:50.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:49:50.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:50.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T12:49:50.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:49:51.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:49:51.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:51.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836487 2026-02-19T12:49:51.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:51.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-133143986178 2026-02-19T12:49:51.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:51.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:49:51.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-133143986178 2026-02-19T12:49:51.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:51.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=133143986178 2026-02-19T12:49:51.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 133143986178' 2026-02-19T12:49:51.428 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 133143986178 2026-02-19T12:49:51.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:49:51.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 133143986178 -lt 133143986178 2026-02-19T12:49:51.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:51.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542150 2026-02-19T12:49:51.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:51.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:49:51.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542150 2026-02-19T12:49:51.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:51.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542150 2026-02-19T12:49:51.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-19T12:49:51.590 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542150 2026-02-19T12:49:51.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:49:51.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-19T12:49:51.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:49:51.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:51.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:51.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:49:51.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:49:51.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:49:51.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:49:51.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:49:51.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:49:51.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:49:51.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:49:52.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:49:52.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:49:52.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:52.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:52.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:49:52.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:49:52.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:49:52.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:358: auto_repair_erasure_coded: get_pg ecpool SOMETHING 2026-02-19T12:49:52.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:49:52.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:49:52.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:49:52.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:49:52.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:358: auto_repair_erasure_coded: local pgid=2.0 2026-02-19T12:49:52.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:359: auto_repair_erasure_coded: get_last_scrub_stamp 2.0 2026-02-19T12:49:52.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:49:52.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:49:52.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:49:52.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:49:52.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:359: auto_repair_erasure_coded: wait_for_scrub 2.0 2026-02-19T12:49:42.279598+0000 2026-02-19T12:49:52.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:49:52.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:49:42.279598+0000 2026-02-19T12:49:52.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:49:52.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:49:52.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:49:52.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:49:52.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:49:52.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:49:52.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:49:52.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:49:52.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:49:51.885815+0000 '>' 2026-02-19T12:49:42.279598+0000 2026-02-19T12:49:52.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:49:52.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:360: auto_repair_erasure_coded: wait_for_clean 2026-02-19T12:49:52.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:49:52.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:49:52.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:49:52.770 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:49:52.771 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:49:52.771 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:49:52.771 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:49:52.771 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:49:52.771 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:49:52.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:49:52.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:49:52.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:49:52.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:49:52.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:49:52.884 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:49:53.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:49:53.041 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:49:53.041 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:49:53.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:49:53.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:53.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:49:53.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836489 2026-02-19T12:49:53.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836489 2026-02-19T12:49:53.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489' 2026-02-19T12:49:53.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:53.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:49:53.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=133143986180 2026-02-19T12:49:53.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 133143986180 2026-02-19T12:49:53.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-133143986180' 2026-02-19T12:49:53.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:53.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:49:53.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542152 2026-02-19T12:49:53.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542152 2026-02-19T12:49:53.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-133143986180 2-60129542152' 2026-02-19T12:49:53.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:53.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836489 2026-02-19T12:49:53.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:53.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:49:53.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836489 2026-02-19T12:49:53.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:53.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836489 2026-02-19T12:49:53.262 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836489 2026-02-19T12:49:53.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-19T12:49:53.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:53.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-19T12:49:53.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:49:54.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:49:54.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:54.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-19T12:49:54.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:49:55.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:49:55.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:55.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836489 2026-02-19T12:49:55.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:55.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-133143986180 2026-02-19T12:49:55.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:55.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:49:55.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-133143986180 2026-02-19T12:49:55.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:55.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=133143986180 2026-02-19T12:49:55.748 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 133143986180 2026-02-19T12:49:55.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 133143986180' 2026-02-19T12:49:55.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:49:55.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 133143986180 -lt 133143986180 2026-02-19T12:49:55.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:55.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542152 2026-02-19T12:49:55.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:55.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:49:55.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542152 2026-02-19T12:49:55.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:55.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542152 2026-02-19T12:49:55.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542152' 2026-02-19T12:49:55.913 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542152 2026-02-19T12:49:55.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:49:56.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542152 -lt 60129542152 2026-02-19T12:49:56.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:49:56.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:56.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:56.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:49:56.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:49:56.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:49:56.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:49:56.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:49:56.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:49:56.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:49:56.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:49:56.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:49:56.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:49:56.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:49:56.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:49:56.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:49:56.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:49:56.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:49:56.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: auto_repair_erasure_coded: get_not_primary ecpool SOMETHING 2026-02-19T12:49:56.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=ecpool 2026-02-19T12:49:56.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:49:56.621 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary ecpool SOMETHING 2026-02-19T12:49:56.621 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=ecpool 2026-02-19T12:49:56.621 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:49:56.621 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:49:56.621 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:49:56.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=2 2026-02-19T12:49:56.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:49:56.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 2)) | .[0]' 2026-02-19T12:49:56.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: auto_repair_erasure_coded: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:49:56.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:49:56.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:49:56.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:49:56.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:49:56.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:49:56.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:49:56.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:49:56.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:49:56.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:49:56.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:49:56.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:49:56.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:49:56.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:49:56.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:49:56.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:49:57.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:49:57.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:49:57.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:49:57.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:49:57.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:49:57.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:49:57.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:49:57.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-19T12:49:57.338 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::deep_scrub_1.3:head#, (61) No data available 2026-02-19T12:49:57.339 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::scrub_1.3:head#, (61) No data available 2026-02-19T12:49:57.339 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#-3:00000000:::deep_scrub_1.0:head#, (61) No data available 2026-02-19T12:49:57.339 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#-3:00000000:::scrub_1.0:head#, (61) No data available 2026-02-19T12:49:57.339 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:49:57.339 INFO:tasks.workunit.client.0.trial144.stdout:hinfo_key 2026-02-19T12:49:57.339 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:49:57.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:49:57.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:49:57.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:49:57.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:49:57.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:49:57.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:49:57.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:49:57.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:49:57.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:49:57.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:49:57.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:49:57.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:49:57.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:49:57.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:49:57.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:49:57.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:49:57.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:49:57.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:49:57.435 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:49:57.435 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:49:57.435 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:49:57.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:57.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:49:57.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:49:57.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:49:57.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:49:57.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:49:57.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:49:57.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:49:57.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:49:57.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:49:57.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:49:57.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:49:57.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:49:57.437 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:49:57.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:49:57.437 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:49:57.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:49:57.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:49:57.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:49:57.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:49:57.456 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:57.453+0000 7f5b45dd4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:57.457 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:57.453+0000 7f5b45dd4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:57.458 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:57.453+0000 7f5b45dd4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:49:57.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:49:57.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:49:57.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:49:57.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:49:57.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:49:57.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:57.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:49:57.591 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:49:57.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:57.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:49:57.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:49:57.845 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:57.841+0000 7f5b45dd4a40 -1 Falling back to public interface 2026-02-19T12:49:58.380 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:49:58.377+0000 7f5b45dd4a40 -1 osd.1 33 log_to_monitors true 2026-02-19T12:49:58.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:49:58.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:49:58.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:49:58.750 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:49:58.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:49:58.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:49:58.911 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 37 up_thru 37 down_at 34 last_clean_interval [31,33) [v2:127.0.0.1:6810/3124138789,v1:127.0.0.1:6811/3124138789] [v2:127.0.0.1:6812/3124138789,v1:127.0.0.1:6813/3124138789] exists,up cf17b739-254d-4d74-9793-8d4e37d2fdda 2026-02-19T12:49:58.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:49:58.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:49:58.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:49:58.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:49:58.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:49:58.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:49:58.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:49:58.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:49:58.913 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:49:58.913 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:49:58.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:49:58.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:49:58.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:49:59.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:49:59.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:49:59.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:49:59.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:49:59.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:49:59.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:49:59.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:49:59.180 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:49:59.180 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:49:59.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:49:59.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:59.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:49:59.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-19T12:49:59.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-19T12:49:59.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491' 2026-02-19T12:49:59.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:59.255 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:49:59.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789954 2026-02-19T12:49:59.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789954 2026-02-19T12:49:59.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-158913789954' 2026-02-19T12:49:59.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:49:59.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:49:59.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542154 2026-02-19T12:49:59.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542154 2026-02-19T12:49:59.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-158913789954 2-60129542154' 2026-02-19T12:49:59.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:49:59.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-19T12:49:59.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:49:59.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:49:59.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836491 2026-02-19T12:49:59.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:49:59.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836491 2026-02-19T12:49:59.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-19T12:49:59.403 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836491 2026-02-19T12:49:59.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:49:59.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T12:49:59.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:50:00.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:50:00.574 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:50:00.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T12:50:00.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:50:01.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:50:01.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:50:01.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836492 -lt 21474836491 2026-02-19T12:50:01.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:01.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-158913789954 2026-02-19T12:50:01.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:01.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:50:01.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-158913789954 2026-02-19T12:50:01.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:01.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789954 2026-02-19T12:50:01.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 158913789954' 2026-02-19T12:50:01.900 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 158913789954 2026-02-19T12:50:01.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:50:02.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789954 -lt 158913789954 2026-02-19T12:50:02.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:02.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542154 2026-02-19T12:50:02.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:02.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:50:02.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542154 2026-02-19T12:50:02.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:02.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542154 2026-02-19T12:50:02.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542154' 2026-02-19T12:50:02.066 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542154 2026-02-19T12:50:02.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:50:02.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542154 -lt 60129542154 2026-02-19T12:50:02.237 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:50:02.237 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:50:02.237 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:50:02.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:50:02.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:50:02.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:50:02.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:50:02.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:50:02.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:50:02.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:50:02.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:50:02.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:50:02.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:50:02.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:50:02.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:50:02.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:50:02.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:50:02.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:50:02.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: auto_repair_erasure_coded: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:50:02.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:365: auto_repair_erasure_coded: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:50:02.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T12:50:02.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:50:02.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:50:02.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:50:02.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:50:02.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:50:02.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:50:02.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:50:02.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:50:02.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:50:02.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:50:02.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:50:02.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:50:02.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:50:02.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:50:02.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:50:02.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:50:02.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:50:02.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:50:02.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:50:02.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:50:02.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:50:02.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:50:02.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:50:02.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:02.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:02.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:50:02.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:50:02.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:50:02.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T12:50:02.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T12:50:02.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T12:50:02.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T12:50:02.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:50:02.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:50:02.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:50:02.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:50:02.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:50:02.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:50:02.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:50:02.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:50:02.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:50:02.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:50:02.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:50:02.952 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:50:02.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:50:02.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:50:02.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:50:02.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:50:02.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:50:02.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:50:02.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:50:02.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:50:02.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:50:02.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:50:02.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:50:02.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:02.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:02.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:50:02.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:50:02.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:50:02.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T12:50:02.962 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:50:02.962 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:02.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:02.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T12:50:02.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:50:02.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T12:50:02.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:50:02.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T12:50:02.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_erasure_coded_overwrites td/osd-scrub-repair 2026-02-19T12:50:02.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:373: TEST_auto_repair_erasure_coded_overwrites: '[' true = true ']' 2026-02-19T12:50:02.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:374: TEST_auto_repair_erasure_coded_overwrites: auto_repair_erasure_coded td/osd-scrub-repair true 2026-02-19T12:50:02.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:328: auto_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-19T12:50:02.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:329: auto_repair_erasure_coded: local allow_overwrites=true 2026-02-19T12:50:02.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:330: auto_repair_erasure_coded: local poolname=ecpool 2026-02-19T12:50:02.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:333: auto_repair_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-19T12:50:02.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T12:50:02.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T12:50:02.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T12:50:02.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T12:50:02.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T12:50:02.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T12:50:02.987 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T12:50:02.987 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:50:02.987 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:50:02.987 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:50:02.987 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:02.987 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:02.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:02.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T12:50:03.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T12:50:03.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T12:50:03.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:50:03.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:50:03.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T12:50:03.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T12:50:03.018 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:50:03.018 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:50:03.019 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:50:03.019 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:50:03.019 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:03.019 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:03.019 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:50:03.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:50:03.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T12:50:03.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T12:50:03.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:50:03.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:50:03.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T12:50:03.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T12:50:03.075 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:50:03.075 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:50:03.076 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:50:03.076 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:50:03.076 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:03.076 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:03.076 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:50:03.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:50:03.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T12:50:03.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:334: auto_repair_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-19T12:50:03.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T12:50:03.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T12:50:03.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T12:50:03.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T12:50:03.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T12:50:03.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T12:50:03.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T12:50:03.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:50:03.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:50:03.226 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:50:03.226 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:03.226 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:03.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:03.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:50:03.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:50:03.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: auto_repair_erasure_coded: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:50:03.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:340: auto_repair_erasure_coded: seq 0 2 2026-02-19T12:50:03.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:340: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-19T12:50:03.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:50:03.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:50:03.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:50:03.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T12:50:03.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:50:03.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:50:03.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:50:03.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:50:03.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:50:03.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:50:03.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:50:03.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:50:03.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:50:03.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:50:03.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:50:03.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:50:03.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:50:03.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:50:03.252 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:50:03.252 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:03.252 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:03.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:03.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:03.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:50:03.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:50:03.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:50:03.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:50:03.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:50:03.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:50:03.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:50:03.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:50:03.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:50:03.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:50:03.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:50:03.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:50:03.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:50:03.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6a927e9b-6b50-4f8e-b8a8-b1e70d7e9c4a 2026-02-19T12:50:03.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 6a927e9b-6b50-4f8e-b8a8-b1e70d7e9c4a' 2026-02-19T12:50:03.255 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 6a927e9b-6b50-4f8e-b8a8-b1e70d7e9c4a 2026-02-19T12:50:03.255 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:50:03.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD7BpdpkWTGDxAAzf13gjUN+Wooqt/8OxQXmw== 2026-02-19T12:50:03.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD7BpdpkWTGDxAAzf13gjUN+Wooqt/8OxQXmw=="}' 2026-02-19T12:50:03.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6a927e9b-6b50-4f8e-b8a8-b1e70d7e9c4a -i td/osd-scrub-repair/0/new.json 2026-02-19T12:50:03.354 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:50:03.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T12:50:03.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQD7BpdpkWTGDxAAzf13gjUN+Wooqt/8OxQXmw== --osd-uuid 6a927e9b-6b50-4f8e-b8a8-b1e70d7e9c4a 2026-02-19T12:50:03.380 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:03.377+0000 7fa02c7b4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:03.382 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:03.377+0000 7fa02c7b4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:03.383 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:03.381+0000 7fa02c7b4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:03.383 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:03.381+0000 7fa02c7b4a40 -1 bdev(0x5644583dcc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:50:03.384 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:03.381+0000 7fa02c7b4a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T12:50:04.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T12:50:04.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:50:04.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T12:50:04.145 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:50:04.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:50:04.353 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:50:04.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T12:50:04.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:50:04.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:50:04.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:50:04.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:50:04.371 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:04.369+0000 7f4fe655ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:04.373 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:04.369+0000 7f4fe655ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:04.373 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:04.369+0000 7f4fe655ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:04.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T12:50:04.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:50:04.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:50:04.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:50:04.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:50:04.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:04.515 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:50:04.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:50:04.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:04.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:50:04.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:50:04.772 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:04.769+0000 7f4fe655ba40 -1 Falling back to public interface 2026-02-19T12:50:05.301 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:05.297+0000 7f4fe655ba40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:50:05.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:50:05.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:05.676 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:50:05.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:50:05.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:05.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:50:05.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:50:06.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:50:06.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:06.838 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:50:06.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:50:06.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:06.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:50:06.994 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3542965448,v1:127.0.0.1:6803/3542965448] [v2:127.0.0.1:6804/3542965448,v1:127.0.0.1:6805/3542965448] exists,up 6a927e9b-6b50-4f8e-b8a8-b1e70d7e9c4a 2026-02-19T12:50:06.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:50:06.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:50:06.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:50:06.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:340: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-19T12:50:06.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:50:06.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:50:06.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:50:06.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T12:50:06.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:50:06.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:50:06.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:50:06.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:50:06.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:50:06.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:50:06.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:50:06.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:50:06.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:50:06.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:50:06.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:50:06.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:50:06.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:50:06.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:50:06.999 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:50:06.999 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:07.000 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:07.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:07.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:07.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:50:07.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:50:07.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:50:07.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:50:07.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:50:07.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:50:07.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:50:07.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:50:07.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:50:07.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:50:07.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:50:07.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:50:07.001 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:50:07.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a3919f5c-53d0-4ded-a7ba-3f28d62b185d 2026-02-19T12:50:07.002 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 a3919f5c-53d0-4ded-a7ba-3f28d62b185d 2026-02-19T12:50:07.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 a3919f5c-53d0-4ded-a7ba-3f28d62b185d' 2026-02-19T12:50:07.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:50:07.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD/BpdpewWoABAAj7k3xt89ISG0ekXfwlZ/zQ== 2026-02-19T12:50:07.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD/BpdpewWoABAAj7k3xt89ISG0ekXfwlZ/zQ=="}' 2026-02-19T12:50:07.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a3919f5c-53d0-4ded-a7ba-3f28d62b185d -i td/osd-scrub-repair/1/new.json 2026-02-19T12:50:07.171 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:50:07.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T12:50:07.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQD/BpdpewWoABAAj7k3xt89ISG0ekXfwlZ/zQ== --osd-uuid a3919f5c-53d0-4ded-a7ba-3f28d62b185d 2026-02-19T12:50:07.198 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:07.193+0000 7f6a083fba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:07.200 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:07.197+0000 7f6a083fba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:07.201 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:07.197+0000 7f6a083fba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:07.202 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:07.197+0000 7f6a083fba40 -1 bdev(0x5625f5b2ec00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:50:07.202 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:07.197+0000 7f6a083fba40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T12:50:07.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T12:50:07.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:50:07.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T12:50:07.827 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:50:07.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:50:08.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T12:50:08.027 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:50:08.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:50:08.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:50:08.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:50:08.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:50:08.045 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:08.041+0000 7f2afcc50a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:08.047 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:08.045+0000 7f2afcc50a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:08.048 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:08.045+0000 7f2afcc50a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:08.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T12:50:08.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:50:08.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:50:08.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:50:08.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:50:08.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:08.191 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:50:08.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:50:08.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:08.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:50:08.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:50:08.453 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:08.449+0000 7f2afcc50a40 -1 Falling back to public interface 2026-02-19T12:50:09.060 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:09.057+0000 7f2afcc50a40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:50:09.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:50:09.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:09.354 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:50:09.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:50:09.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:09.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:50:09.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:50:10.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:50:10.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:10.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:50:10.514 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:50:10.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:10.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:50:10.669 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/555353465,v1:127.0.0.1:6811/555353465] [v2:127.0.0.1:6812/555353465,v1:127.0.0.1:6813/555353465] exists,up a3919f5c-53d0-4ded-a7ba-3f28d62b185d 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:340: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:50:10.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:50:10.671 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:50:10.671 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:50:10.671 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:10.671 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:10.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:10.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:10.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:50:10.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:50:10.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:50:10.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:50:10.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:50:10.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:50:10.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:50:10.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:50:10.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:50:10.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:50:10.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:50:10.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:50:10.673 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:50:10.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=80c180b7-1da5-4091-9ed0-db6178fe3f21 2026-02-19T12:50:10.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 80c180b7-1da5-4091-9ed0-db6178fe3f21' 2026-02-19T12:50:10.674 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 80c180b7-1da5-4091-9ed0-db6178fe3f21 2026-02-19T12:50:10.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:50:10.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQACB5dpYJDtKBAA5n1f2yTyzng8m7pCBNxGGg== 2026-02-19T12:50:10.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQACB5dpYJDtKBAA5n1f2yTyzng8m7pCBNxGGg=="}' 2026-02-19T12:50:10.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 80c180b7-1da5-4091-9ed0-db6178fe3f21 -i td/osd-scrub-repair/2/new.json 2026-02-19T12:50:10.848 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:50:10.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T12:50:10.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQACB5dpYJDtKBAA5n1f2yTyzng8m7pCBNxGGg== --osd-uuid 80c180b7-1da5-4091-9ed0-db6178fe3f21 2026-02-19T12:50:10.877 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:10.873+0000 7f79cdf5fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:10.878 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:10.873+0000 7f79cdf5fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:10.879 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:10.877+0000 7f79cdf5fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:10.880 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:10.877+0000 7f79cdf5fa40 -1 bdev(0x562048c70c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T12:50:10.880 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:10.877+0000 7f79cdf5fa40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T12:50:11.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T12:50:11.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:50:11.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T12:50:11.515 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T12:50:11.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:50:11.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T12:50:11.716 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T12:50:11.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:50:11.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:50:11.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:50:11.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:50:11.742 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:11.737+0000 7f7ebf1b6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:11.743 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:11.741+0000 7f7ebf1b6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:11.745 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:11.741+0000 7f7ebf1b6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:11.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T12:50:11.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:50:11.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:50:11.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:50:11.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:50:11.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:11.875 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:50:11.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:50:11.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:11.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:50:12.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:50:12.153 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:12.149+0000 7f7ebf1b6a40 -1 Falling back to public interface 2026-02-19T12:50:12.789 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:12.785+0000 7f7ebf1b6a40 -1 osd.2 0 log_to_monitors true 2026-02-19T12:50:13.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:50:13.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:13.032 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:50:13.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:50:13.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:13.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:50:13.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:50:14.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:50:14.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:14.191 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:50:14.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:50:14.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:14.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:50:14.347 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2886990262,v1:127.0.0.1:6819/2886990262] [v2:127.0.0.1:6820/2886990262,v1:127.0.0.1:6821/2886990262] exists,up 80c180b7-1da5-4091-9ed0-db6178fe3f21 2026-02-19T12:50:14.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:50:14.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:50:14.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:50:14.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:343: auto_repair_erasure_coded: create_rbd_pool 2026-02-19T12:50:14.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T12:50:14.494 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T12:50:14.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T12:50:14.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T12:50:14.720 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T12:50:14.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:50:15.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T12:50:16.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:344: auto_repair_erasure_coded: wait_for_clean 2026-02-19T12:50:16.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:50:16.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:50:16.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:50:16.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:50:16.014 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:50:16.014 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:50:16.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:50:16.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:50:16.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:50:16.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:50:16.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:50:16.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:50:16.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:50:16.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:50:16.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:50:16.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:50:16.285 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:50:16.285 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:50:16.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:50:16.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:50:16.285 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:50:16.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T12:50:16.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T12:50:16.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T12:50:16.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:50:16.357 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:50:16.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-19T12:50:16.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-19T12:50:16.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705667' 2026-02-19T12:50:16.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:50:16.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:50:16.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542146 2026-02-19T12:50:16.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542146 2026-02-19T12:50:16.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705667 2-60129542146' 2026-02-19T12:50:16.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:16.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T12:50:16.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:16.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:50:16.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T12:50:16.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:16.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T12:50:16.510 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T12:50:16.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T12:50:16.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:50:16.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T12:50:16.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:50:17.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:50:17.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:50:17.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836483 2026-02-19T12:50:17.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:17.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-19T12:50:17.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:17.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:50:17.827 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705667 2026-02-19T12:50:17.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:17.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705667 2026-02-19T12:50:17.829 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705667 2026-02-19T12:50:17.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-19T12:50:17.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:50:17.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705667 -lt 38654705667 2026-02-19T12:50:17.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:17.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542146 2026-02-19T12:50:17.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:18.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:50:18.001 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542146 2026-02-19T12:50:18.001 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:18.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542146 2026-02-19T12:50:18.003 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542146 2026-02-19T12:50:18.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-19T12:50:18.003 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:50:18.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542146 -lt 60129542146 2026-02-19T12:50:18.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:50:18.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:50:18.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:50:18.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:50:18.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:50:18.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:50:18.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:50:18.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:50:18.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:50:18.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:50:18.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:50:18.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:50:18.514 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:50:18.514 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:50:18.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:50:18.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:50:18.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:50:18.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:50:18.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:347: auto_repair_erasure_coded: create_ec_pool ecpool true k=2 m=1 2026-02-19T12:50:18.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: create_ec_pool: local pool_name=ecpool 2026-02-19T12:50:18.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: create_ec_pool: shift 2026-02-19T12:50:18.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: create_ec_pool: local allow_overwrites=true 2026-02-19T12:50:18.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2525: create_ec_pool: shift 2026-02-19T12:50:18.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-19T12:50:18.903 INFO:tasks.workunit.client.0.trial144.stderr:load: isa 2026-02-19T12:50:18.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-19T12:50:18.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-19T12:50:19.208 INFO:tasks.workunit.client.0.trial144.stderr:pool 'ecpool' created 2026-02-19T12:50:19.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:50:20.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: create_ec_pool: '[' true = true ']' 2026-02-19T12:50:20.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2532: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-19T12:50:20.400 INFO:tasks.workunit.client.0.trial144.stderr:set pool 2 allow_ec_overwrites to true 2026-02-19T12:50:20.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2535: create_ec_pool: wait_for_clean 2026-02-19T12:50:20.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:50:20.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:50:20.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:50:20.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:50:20.413 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:50:20.414 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:50:20.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:50:20.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:50:20.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:50:20.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:50:20.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:50:20.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:50:20.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:50:20.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:50:20.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:50:20.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:50:20.691 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:50:20.692 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:50:20.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:50:20.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:50:20.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:50:20.764 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-19T12:50:20.764 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-19T12:50:20.764 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485' 2026-02-19T12:50:20.764 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:50:20.764 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:50:20.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705669 2026-02-19T12:50:20.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705669 2026-02-19T12:50:20.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-38654705669' 2026-02-19T12:50:20.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:50:20.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:50:20.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542148 2026-02-19T12:50:20.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542148 2026-02-19T12:50:20.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-38654705669 2-60129542148' 2026-02-19T12:50:20.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:20.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-19T12:50:20.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:20.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:50:20.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836485 2026-02-19T12:50:20.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:20.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836485 2026-02-19T12:50:20.914 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836485 2026-02-19T12:50:20.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-19T12:50:20.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:50:21.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-19T12:50:21.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:50:22.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:50:22.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:50:22.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-19T12:50:22.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:22.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705669 2026-02-19T12:50:22.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:22.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:50:22.249 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705669 2026-02-19T12:50:22.249 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:22.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705669 2026-02-19T12:50:22.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-19T12:50:22.251 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705669 2026-02-19T12:50:22.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:50:22.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-19T12:50:22.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:22.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542148 2026-02-19T12:50:22.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:22.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:50:22.416 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542148 2026-02-19T12:50:22.417 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:22.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542148 2026-02-19T12:50:22.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-19T12:50:22.418 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542148 2026-02-19T12:50:22.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:50:22.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-19T12:50:22.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:50:22.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:50:22.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:50:22.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:50:22.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:50:22.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:50:22.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:50:22.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:50:22.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:50:22.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:50:22.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:50:22.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:50:22.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:50:22.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:50:22.934 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:50:23.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:50:23.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:50:23.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:50:23.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2536: create_ec_pool: return 0 2026-02-19T12:50:23.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:350: auto_repair_erasure_coded: local payload=ABCDEF 2026-02-19T12:50:23.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:351: auto_repair_erasure_coded: echo ABCDEF 2026-02-19T12:50:23.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: auto_repair_erasure_coded: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-19T12:50:23.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:356: auto_repair_erasure_coded: get_not_primary ecpool SOMETHING 2026-02-19T12:50:23.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=ecpool 2026-02-19T12:50:23.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:50:23.145 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary ecpool SOMETHING 2026-02-19T12:50:23.145 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=ecpool 2026-02-19T12:50:23.145 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:50:23.145 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:50:23.145 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:50:23.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=2 2026-02-19T12:50:23.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:50:23.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 2)) | .[0]' 2026-02-19T12:50:23.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:356: auto_repair_erasure_coded: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:50:23.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:50:23.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:50:23.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:50:23.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:50:23.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:50:23.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:50:23.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:50:23.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:50:23.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:50:23.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:50:23.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:50:23.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:50:23.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:50:23.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:50:23.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:50:23.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:50:23.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:50:23.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:50:23.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:50:23.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:50:23.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:50:23.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:50:23.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-19T12:50:23.955 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::deep_scrub_1.3:head#, (61) No data available 2026-02-19T12:50:23.955 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::scrub_1.3:head#, (61) No data available 2026-02-19T12:50:23.956 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#-3:00000000:::deep_scrub_1.0:head#, (61) No data available 2026-02-19T12:50:23.956 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#-3:00000000:::scrub_1.0:head#, (61) No data available 2026-02-19T12:50:23.956 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-19T12:50:24.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:50:24.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:50:24.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:50:24.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:50:24.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:50:24.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:50:24.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:50:24.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:50:24.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:50:24.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:50:24.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:50:24.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:50:24.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:50:24.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:50:24.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:50:24.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:50:24.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:50:24.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:50:24.063 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:50:24.063 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:24.064 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:24.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:24.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:24.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:50:24.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:50:24.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:50:24.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:50:24.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:50:24.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:50:24.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:50:24.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:50:24.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:50:24.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:50:24.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:50:24.066 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:50:24.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:50:24.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:50:24.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:50:24.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:50:24.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:50:24.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:50:24.084 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:24.080+0000 7f75b9eaea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:24.085 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:24.080+0000 7f75b9eaea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:24.086 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:24.080+0000 7f75b9eaea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:24.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:50:24.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:50:24.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:50:24.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:50:24.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:50:24.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:24.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:50:24.221 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:50:24.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:24.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:50:24.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:50:24.477 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:24.472+0000 7f75b9eaea40 -1 Falling back to public interface 2026-02-19T12:50:25.038 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:25.032+0000 7f75b9eaea40 -1 osd.1 28 log_to_monitors true 2026-02-19T12:50:25.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:50:25.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:25.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:50:25.383 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:50:25.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:25.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:50:25.542 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 32 up_thru 32 down_at 29 last_clean_interval [9,28) [v2:127.0.0.1:6810/1839074465,v1:127.0.0.1:6811/1839074465] [v2:127.0.0.1:6812/1839074465,v1:127.0.0.1:6813/1839074465] exists,up a3919f5c-53d0-4ded-a7ba-3f28d62b185d 2026-02-19T12:50:25.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:50:25.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:50:25.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:50:25.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:50:25.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:50:25.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:50:25.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:50:25.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:50:25.543 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:50:25.543 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:50:25.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:50:25.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:50:25.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:50:25.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:50:25.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:50:25.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:50:25.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:50:25.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:50:25.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:50:25.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:50:25.816 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:50:25.816 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:50:25.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:50:25.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:50:25.816 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:50:25.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-19T12:50:25.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-19T12:50:25.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-19T12:50:25.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:50:25.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:50:25.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=137438953474 2026-02-19T12:50:25.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 137438953474 2026-02-19T12:50:25.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-137438953474' 2026-02-19T12:50:25.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:50:25.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:50:26.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542150 2026-02-19T12:50:26.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542150 2026-02-19T12:50:26.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-137438953474 2-60129542150' 2026-02-19T12:50:26.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:26.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-19T12:50:26.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:26.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:50:26.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-19T12:50:26.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:26.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-19T12:50:26.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-19T12:50:26.039 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836487 2026-02-19T12:50:26.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:50:26.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T12:50:26.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:50:27.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:50:27.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:50:27.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T12:50:27.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:50:28.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:50:28.366 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:50:28.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836487 2026-02-19T12:50:28.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:28.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-137438953474 2026-02-19T12:50:28.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:28.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:50:28.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-137438953474 2026-02-19T12:50:28.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:28.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=137438953474 2026-02-19T12:50:28.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 137438953474' 2026-02-19T12:50:28.533 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 137438953474 2026-02-19T12:50:28.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:50:28.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 137438953474 -lt 137438953474 2026-02-19T12:50:28.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:28.697 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542150 2026-02-19T12:50:28.697 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:28.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:50:28.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542150 2026-02-19T12:50:28.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:28.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542150 2026-02-19T12:50:28.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-19T12:50:28.700 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542150 2026-02-19T12:50:28.701 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:50:28.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-19T12:50:28.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:50:28.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:50:28.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:50:29.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:50:29.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:50:29.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:50:29.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:50:29.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:50:29.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:50:29.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:50:29.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:50:29.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:50:29.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:50:29.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:50:29.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:50:29.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:50:29.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:50:29.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:50:29.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:358: auto_repair_erasure_coded: get_pg ecpool SOMETHING 2026-02-19T12:50:29.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:50:29.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:50:29.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:50:29.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:50:29.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:358: auto_repair_erasure_coded: local pgid=2.0 2026-02-19T12:50:29.574 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:359: auto_repair_erasure_coded: get_last_scrub_stamp 2.0 2026-02-19T12:50:29.574 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:29.575 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:29.575 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:29.575 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:29.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:359: auto_repair_erasure_coded: wait_for_scrub 2.0 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:29.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:50:29.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:29.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:50:29.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:50:29.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:50:29.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:50:29.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:29.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:29.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:29.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:29.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:50:28.809670+0000 '>' 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:29.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:50:30.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:50:30.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:50:30.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:50:30.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:30.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:30.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:30.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:31.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:50:28.809670+0000 '>' 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:31.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:50:32.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:50:32.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:50:32.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:50:32.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:32.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:32.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:32.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:32.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:50:28.809670+0000 '>' 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:32.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:50:33.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:50:33.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:50:33.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:50:33.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:33.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:33.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:33.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:33.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:50:28.809670+0000 '>' 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:33.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:50:34.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:50:34.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:50:34.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:50:34.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:34.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:34.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:34.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:34.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:50:28.809670+0000 '>' 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:34.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:50:35.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:50:35.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:50:35.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:50:35.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:35.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:35.536 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:35.536 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:35.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:50:28.809670+0000 '>' 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:35.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:50:36.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:50:36.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:50:36.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:50:36.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:36.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:36.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:36.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:36.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:50:28.809670+0000 '>' 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:36.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:50:37.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:50:37.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:50:37.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:50:37.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:37.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:37.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:37.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:38.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:50:28.809670+0000 '>' 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:38.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:50:39.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:50:39.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:50:39.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:50:39.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:39.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:39.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:39.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:39.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:50:28.809670+0000 '>' 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:39.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:50:40.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:50:40.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:50:40.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:50:40.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:40.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:40.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:40.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:40.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:50:28.809670+0000 '>' 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:40.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:50:41.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:50:41.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:50:41.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:50:41.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:41.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:41.328 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:41.328 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:41.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:50:28.809670+0000 '>' 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:41.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:50:42.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:50:42.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:50:42.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:50:42.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:42.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:42.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:42.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:42.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:50:28.809670+0000 '>' 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:42.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:50:43.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:50:43.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:50:43.651 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:50:43.651 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:43.651 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:43.651 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:43.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:43.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:50:28.809670+0000 '>' 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:43.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:50:44.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:50:44.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:50:44.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:50:44.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:44.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:44.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:44.811 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:44.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:50:28.809670+0000 '>' 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:44.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:50:45.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:50:45.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:50:45.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:50:45.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:50:45.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:50:45.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:50:45.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:50:46.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:50:42.697399+0000 '>' 2026-02-19T12:50:28.809670+0000 2026-02-19T12:50:46.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:50:46.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:360: auto_repair_erasure_coded: wait_for_clean 2026-02-19T12:50:46.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:50:46.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:50:46.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:50:46.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:50:46.127 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:50:46.127 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:50:46.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:50:46.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:50:46.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:50:46.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:50:46.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:50:46.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:50:46.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:50:46.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:50:46.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:50:46.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:50:46.398 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:50:46.398 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:50:46.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:50:46.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:50:46.398 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:50:46.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836493 2026-02-19T12:50:46.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836493 2026-02-19T12:50:46.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493' 2026-02-19T12:50:46.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:50:46.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:50:46.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=137438953479 2026-02-19T12:50:46.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 137438953479 2026-02-19T12:50:46.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-137438953479' 2026-02-19T12:50:46.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:50:46.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:50:46.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542155 2026-02-19T12:50:46.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542155 2026-02-19T12:50:46.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-137438953479 2-60129542155' 2026-02-19T12:50:46.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:46.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836493 2026-02-19T12:50:46.615 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:46.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:50:46.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836493 2026-02-19T12:50:46.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:46.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836493 2026-02-19T12:50:46.619 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836493 2026-02-19T12:50:46.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-19T12:50:46.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:50:46.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836491 -lt 21474836493 2026-02-19T12:50:46.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:50:47.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:50:47.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:50:47.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836493 -lt 21474836493 2026-02-19T12:50:47.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:47.939 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-137438953479 2026-02-19T12:50:47.939 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:47.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:50:47.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-137438953479 2026-02-19T12:50:47.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:47.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=137438953479 2026-02-19T12:50:47.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 137438953479' 2026-02-19T12:50:47.943 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 137438953479 2026-02-19T12:50:47.943 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:50:48.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 137438953479 -lt 137438953479 2026-02-19T12:50:48.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:48.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542155 2026-02-19T12:50:48.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:48.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:50:48.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542155 2026-02-19T12:50:48.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:48.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542155 2026-02-19T12:50:48.116 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542155 2026-02-19T12:50:48.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542155' 2026-02-19T12:50:48.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:50:48.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542155 -lt 60129542155 2026-02-19T12:50:48.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:50:48.287 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:50:48.287 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:50:48.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:50:48.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:50:48.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:50:48.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:50:48.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:50:48.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:50:48.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:50:48.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:50:48.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:50:48.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:50:48.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:50:48.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:50:48.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:50:48.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:50:48.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:50:48.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: auto_repair_erasure_coded: get_not_primary ecpool SOMETHING 2026-02-19T12:50:48.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=ecpool 2026-02-19T12:50:48.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:50:48.833 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary ecpool SOMETHING 2026-02-19T12:50:48.834 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=ecpool 2026-02-19T12:50:48.834 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:50:48.834 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:50:48.834 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:50:48.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=2 2026-02-19T12:50:48.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:50:48.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 2)) | .[0]' 2026-02-19T12:50:49.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: auto_repair_erasure_coded: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:50:49.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:50:49.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:50:49.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:50:49.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:50:49.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:50:49.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:50:49.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:50:49.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:50:49.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:50:49.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:50:49.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:50:49.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:50:49.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:50:49.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:50:49.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:50:49.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:50:49.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:50:49.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:50:49.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:50:49.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:50:49.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:50:49.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:50:49.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-19T12:50:49.530 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:50:49.530 INFO:tasks.workunit.client.0.trial144.stdout:hinfo_key 2026-02-19T12:50:49.530 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:50:49.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:50:49.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:50:49.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:50:49.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:50:49.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:50:49.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:50:49.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:50:49.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:50:49.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:50:49.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:50:49.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:50:49.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:50:49.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:50:49.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:50:49.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:50:49.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:50:49.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:50:49.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:50:49.643 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:50:49.643 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:49.643 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:49.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:50:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:50:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:50:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:50:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:50:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:50:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:50:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:50:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T12:50:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:50:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:50:49.645 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:50:49.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-19T12:50:49.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:50:49.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:50:49.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:50:49.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:50:49.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:50:49.661 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:49.656+0000 7f97aabd0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:49.662 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:49.656+0000 7f97aabd0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:49.663 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:49.660+0000 7f97aabd0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:49.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:50:49.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:50:49.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:50:49.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:50:49.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:50:49.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:49.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:50:49.807 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:50:49.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:49.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:50:49.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:50:50.068 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:50.064+0000 7f97aabd0a40 -1 Falling back to public interface 2026-02-19T12:50:50.754 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:50.752+0000 7f97aabd0a40 -1 osd.1 36 log_to_monitors true 2026-02-19T12:50:50.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:50:50.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:50.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:50:50.964 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:50:50.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:50.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:50:51.122 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [32,36) [v2:127.0.0.1:6810/1206564199,v1:127.0.0.1:6811/1206564199] [v2:127.0.0.1:6812/1206564199,v1:127.0.0.1:6813/1206564199] exists,up a3919f5c-53d0-4ded-a7ba-3f28d62b185d 2026-02-19T12:50:51.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:50:51.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:50:51.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:50:51.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:50:51.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:50:51.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:50:51.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:50:51.123 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:50:51.123 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:50:51.123 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:50:51.123 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:50:51.123 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:50:51.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:50:51.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:50:51.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:50:51.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:50:51.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:50:51.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:50:51.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:50:51.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:50:51.390 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:50:51.390 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:50:51.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:50:51.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:50:51.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:50:51.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836494 2026-02-19T12:50:51.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836494 2026-02-19T12:50:51.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836494' 2026-02-19T12:50:51.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:50:51.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:50:51.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724546 2026-02-19T12:50:51.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724546 2026-02-19T12:50:51.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836494 1-167503724546' 2026-02-19T12:50:51.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:50:51.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:50:51.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542157 2026-02-19T12:50:51.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542157 2026-02-19T12:50:51.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836494 1-167503724546 2-60129542157' 2026-02-19T12:50:51.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:51.606 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836494 2026-02-19T12:50:51.606 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:51.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:50:51.609 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836494 2026-02-19T12:50:51.609 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:51.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836494 2026-02-19T12:50:51.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836494' 2026-02-19T12:50:51.610 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836494 2026-02-19T12:50:51.610 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:50:51.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836493 -lt 21474836494 2026-02-19T12:50:51.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:50:52.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:50:52.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:50:52.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836495 -lt 21474836494 2026-02-19T12:50:52.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:52.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-167503724546 2026-02-19T12:50:52.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:52.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:50:52.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-167503724546 2026-02-19T12:50:52.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:52.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724546 2026-02-19T12:50:52.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 167503724546' 2026-02-19T12:50:52.942 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 167503724546 2026-02-19T12:50:52.942 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:50:53.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724546 -lt 167503724546 2026-02-19T12:50:53.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:50:53.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542157 2026-02-19T12:50:53.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:50:53.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:50:53.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542157 2026-02-19T12:50:53.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:50:53.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542157 2026-02-19T12:50:53.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542157' 2026-02-19T12:50:53.116 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542157 2026-02-19T12:50:53.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:50:53.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542157 -lt 60129542157 2026-02-19T12:50:53.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:50:53.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:50:53.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:50:53.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:50:53.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:50:53.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:50:53.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:50:53.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:50:53.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:50:53.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:50:53.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:50:53.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:50:53.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:50:53.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:50:53.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:50:53.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:50:53.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:50:53.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:50:53.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: auto_repair_erasure_coded: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:50:53.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:365: auto_repair_erasure_coded: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:50:53.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T12:50:53.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:50:53.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:50:53.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:50:53.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:50:53.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:50:53.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:50:53.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:50:53.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:50:54.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:50:54.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:50:54.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:50:54.156 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:50:54.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:50:54.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:50:54.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:50:54.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:50:54.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:50:54.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:50:54.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:50:54.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:50:54.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:50:54.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:50:54.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:50:54.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:54.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:54.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:50:54.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:50:54.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:50:54.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T12:50:54.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T12:50:54.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T12:50:54.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T12:50:54.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:50:54.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:50:54.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:50:54.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:50:54.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:50:54.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:50:54.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:50:54.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:50:54.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:50:54.183 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:50:54.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:50:54.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:50:54.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:50:54.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:50:54.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:50:54.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:50:54.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:50:54.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:50:54.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:50:54.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:50:54.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:50:54.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:50:54.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:50:54.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:54.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:54.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:50:54.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:50:54.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:50:54.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T12:50:54.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:50:54.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:54.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:54.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T12:50:54.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:50:54.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T12:50:54.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:50:54.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T12:50:54.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_and_repair_jerasure_appends td/osd-scrub-repair 2026-02-19T12:50:54.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:874: TEST_corrupt_and_repair_jerasure_appends: corrupt_and_repair_jerasure td/osd-scrub-repair false 2026-02-19T12:50:54.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:857: corrupt_and_repair_jerasure: local dir=td/osd-scrub-repair 2026-02-19T12:50:54.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:858: corrupt_and_repair_jerasure: local allow_overwrites=false 2026-02-19T12:50:54.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: corrupt_and_repair_jerasure: local poolname=ecpool 2026-02-19T12:50:54.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:861: corrupt_and_repair_jerasure: run_mon td/osd-scrub-repair a 2026-02-19T12:50:54.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T12:50:54.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T12:50:54.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T12:50:54.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T12:50:54.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T12:50:54.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T12:50:54.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T12:50:54.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:50:54.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:50:54.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:50:54.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:54.221 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:54.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:54.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T12:50:54.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T12:50:54.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T12:50:54.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:50:54.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:50:54.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T12:50:54.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T12:50:54.253 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:50:54.253 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:50:54.253 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:50:54.254 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:50:54.254 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:54.254 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:54.254 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:50:54.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:50:54.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T12:50:54.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T12:50:54.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:50:54.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:50:54.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T12:50:54.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T12:50:54.310 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:50:54.310 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:50:54.311 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:50:54.311 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:50:54.311 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:54.311 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:54.311 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:50:54.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:50:54.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T12:50:54.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:862: corrupt_and_repair_jerasure: run_mgr td/osd-scrub-repair x 2026-02-19T12:50:54.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T12:50:54.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T12:50:54.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T12:50:54.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T12:50:54.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T12:50:54.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T12:50:54.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T12:50:54.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:50:54.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:50:54.464 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:50:54.464 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:54.464 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:54.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:54.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:50:54.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:50:54.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:863: corrupt_and_repair_jerasure: seq 0 3 2026-02-19T12:50:54.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:863: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-19T12:50:54.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:864: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 0 2026-02-19T12:50:54.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:50:54.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:50:54.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T12:50:54.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:50:54.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:50:54.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:50:54.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:50:54.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:50:54.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:50:54.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:50:54.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:50:54.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:50:54.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:50:54.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:50:54.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:50:54.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:50:54.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:50:54.489 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:50:54.489 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:54.489 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:54.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:54.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:54.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:50:54.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:50:54.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:50:54.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:50:54.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:50:54.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:50:54.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:50:54.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:50:54.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:50:54.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:50:54.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:50:54.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:50:54.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:50:54.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=afa1364e-cd78-4f69-9483-463fc9f5e9d1 2026-02-19T12:50:54.491 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 afa1364e-cd78-4f69-9483-463fc9f5e9d1 2026-02-19T12:50:54.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 afa1364e-cd78-4f69-9483-463fc9f5e9d1' 2026-02-19T12:50:54.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:50:54.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAuB5dp0InXHRAAOnSPLZPs2CQdD/0bg3FNXg== 2026-02-19T12:50:54.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAuB5dp0InXHRAAOnSPLZPs2CQdD/0bg3FNXg=="}' 2026-02-19T12:50:54.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new afa1364e-cd78-4f69-9483-463fc9f5e9d1 -i td/osd-scrub-repair/0/new.json 2026-02-19T12:50:54.588 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:50:54.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T12:50:54.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAuB5dp0InXHRAAOnSPLZPs2CQdD/0bg3FNXg== --osd-uuid afa1364e-cd78-4f69-9483-463fc9f5e9d1 2026-02-19T12:50:54.612 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:54.612+0000 7ff5621f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:54.614 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:54.612+0000 7ff5621f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:54.615 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:54.616+0000 7ff5621f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:54.615 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:54.616+0000 7ff5621f2a40 -1 bdev(0x559113b98c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:50:54.615 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:54.616+0000 7ff5621f2a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T12:50:55.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T12:50:55.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:50:55.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T12:50:55.382 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:50:55.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:50:55.589 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:50:55.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T12:50:55.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:50:55.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:50:55.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:50:55.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:50:55.614 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:55.612+0000 7fac80b66a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:55.615 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:55.616+0000 7fac80b66a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:55.616 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:55.616+0000 7fac80b66a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:55.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T12:50:55.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:50:55.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:50:55.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:50:55.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:50:55.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:55.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:50:55.749 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:50:55.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:55.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:50:55.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:50:56.000 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:56.000+0000 7fac80b66a40 -1 Falling back to public interface 2026-02-19T12:50:56.637 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:56.636+0000 7fac80b66a40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:50:56.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:50:56.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:56.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:50:56.907 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:50:56.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:56.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:50:57.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:50:58.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:50:58.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:58.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:50:58.073 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:50:58.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:58.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:50:58.233 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4179791944,v1:127.0.0.1:6803/4179791944] [v2:127.0.0.1:6804/4179791944,v1:127.0.0.1:6805/4179791944] exists,up afa1364e-cd78-4f69-9483-463fc9f5e9d1 2026-02-19T12:50:58.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:50:58.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:50:58.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:50:58.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:863: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-19T12:50:58.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:864: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 1 2026-02-19T12:50:58.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:50:58.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:50:58.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T12:50:58.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:50:58.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:50:58.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:50:58.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:50:58.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:50:58.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:50:58.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:50:58.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:50:58.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:50:58.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:50:58.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:50:58.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:50:58.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:50:58.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:50:58.238 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:50:58.238 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:50:58.239 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:50:58.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:58.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:50:58.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:50:58.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:50:58.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:50:58.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:50:58.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:50:58.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:50:58.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:50:58.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:50:58.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:50:58.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:50:58.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:50:58.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:50:58.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:50:58.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6a06dd6f-2a04-4584-96dd-5506f512ce78 2026-02-19T12:50:58.241 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 6a06dd6f-2a04-4584-96dd-5506f512ce78 2026-02-19T12:50:58.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 6a06dd6f-2a04-4584-96dd-5506f512ce78' 2026-02-19T12:50:58.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:50:58.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAyB5dpgcnqDhAAZfi8IWrAsVs7Vu/awVaunw== 2026-02-19T12:50:58.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAyB5dpgcnqDhAAZfi8IWrAsVs7Vu/awVaunw=="}' 2026-02-19T12:50:58.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6a06dd6f-2a04-4584-96dd-5506f512ce78 -i td/osd-scrub-repair/1/new.json 2026-02-19T12:50:58.413 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:50:58.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T12:50:58.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAyB5dpgcnqDhAAZfi8IWrAsVs7Vu/awVaunw== --osd-uuid 6a06dd6f-2a04-4584-96dd-5506f512ce78 2026-02-19T12:50:58.440 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:58.440+0000 7f31b50b7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:58.442 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:58.440+0000 7f31b50b7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:58.443 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:58.444+0000 7f31b50b7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:58.444 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:58.444+0000 7f31b50b7a40 -1 bdev(0x564951526c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:50:58.444 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:58.444+0000 7f31b50b7a40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T12:50:59.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T12:50:59.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:50:59.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T12:50:59.099 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:50:59.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:50:59.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T12:50:59.306 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:50:59.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:50:59.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:50:59.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:50:59.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:50:59.322 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:59.320+0000 7f7eabea0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:59.323 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:59.324+0000 7f7eabea0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:59.324 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:59.324+0000 7f7eabea0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:50:59.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T12:50:59.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:50:59.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:50:59.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:50:59.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:50:59.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:50:59.463 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:50:59.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:50:59.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:50:59.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:50:59.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:50:59.676 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:50:59.676+0000 7f7eabea0a40 -1 Falling back to public interface 2026-02-19T12:51:00.541 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:00.540+0000 7f7eabea0a40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:51:00.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:51:00.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:00.624 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:51:00.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:51:00.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:00.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:51:00.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:51:01.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:51:01.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:01.783 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:51:01.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:51:01.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:01.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:51:01.941 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2365704413,v1:127.0.0.1:6811/2365704413] [v2:127.0.0.1:6812/2365704413,v1:127.0.0.1:6813/2365704413] exists,up 6a06dd6f-2a04-4584-96dd-5506f512ce78 2026-02-19T12:51:01.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:51:01.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:51:01.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:51:01.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:863: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-19T12:51:01.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:864: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 2 2026-02-19T12:51:01.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:51:01.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:51:01.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T12:51:01.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:51:01.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:51:01.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:51:01.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:51:01.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:51:01.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:51:01.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:51:01.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:51:01.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:51:01.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:51:01.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:51:01.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:51:01.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:51:01.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:51:01.945 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:51:01.945 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:51:01.945 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:51:01.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:01.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:01.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:51:01.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:51:01.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:51:01.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:51:01.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:51:01.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:51:01.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:51:01.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:51:01.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:51:01.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:51:01.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:51:01.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:51:01.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:51:01.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=56b9b2cb-98f6-4b89-81c3-a3d557f1da96 2026-02-19T12:51:01.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 56b9b2cb-98f6-4b89-81c3-a3d557f1da96' 2026-02-19T12:51:01.947 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 56b9b2cb-98f6-4b89-81c3-a3d557f1da96 2026-02-19T12:51:01.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:51:01.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA1B5dpAMwiORAAPllIOrU53j3j7G1ksSVmYQ== 2026-02-19T12:51:01.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA1B5dpAMwiORAAPllIOrU53j3j7G1ksSVmYQ=="}' 2026-02-19T12:51:01.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 56b9b2cb-98f6-4b89-81c3-a3d557f1da96 -i td/osd-scrub-repair/2/new.json 2026-02-19T12:51:02.123 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:51:02.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T12:51:02.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA1B5dpAMwiORAAPllIOrU53j3j7G1ksSVmYQ== --osd-uuid 56b9b2cb-98f6-4b89-81c3-a3d557f1da96 2026-02-19T12:51:02.149 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:02.148+0000 7fc325ba2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:02.151 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:02.152+0000 7fc325ba2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:02.152 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:02.152+0000 7fc325ba2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:02.152 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:02.152+0000 7fc325ba2a40 -1 bdev(0x55879e4bcc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T12:51:02.153 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:02.152+0000 7fc325ba2a40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T12:51:02.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T12:51:02.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:51:02.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T12:51:02.795 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T12:51:02.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:51:03.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T12:51:03.000 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T12:51:03.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:51:03.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:51:03.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:51:03.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:51:03.024 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:03.024+0000 7fe02653ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:03.026 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:03.024+0000 7fe02653ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:03.027 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:03.028+0000 7fe02653ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:03.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T12:51:03.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:51:03.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:51:03.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:51:03.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:51:03.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:03.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:51:03.156 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:51:03.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:03.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:51:03.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:51:03.404 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:03.404+0000 7fe02653ea40 -1 Falling back to public interface 2026-02-19T12:51:04.081 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:04.080+0000 7fe02653ea40 -1 osd.2 0 log_to_monitors true 2026-02-19T12:51:04.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:51:04.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:04.317 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:51:04.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:51:04.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:04.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:51:04.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:51:05.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:51:05.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:05.476 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:51:05.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:51:05.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:05.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:51:05.633 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2176959250,v1:127.0.0.1:6819/2176959250] [v2:127.0.0.1:6820/2176959250,v1:127.0.0.1:6821/2176959250] exists,up 56b9b2cb-98f6-4b89-81c3-a3d557f1da96 2026-02-19T12:51:05.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:51:05.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:51:05.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:51:05.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:863: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-19T12:51:05.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:864: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 3 2026-02-19T12:51:05.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:51:05.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:51:05.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T12:51:05.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:51:05.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:51:05.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:51:05.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:51:05.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:51:05.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:51:05.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:51:05.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:51:05.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:51:05.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:51:05.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:51:05.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:51:05.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:51:05.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:51:05.638 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:51:05.638 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:51:05.638 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:51:05.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:05.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:05.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:51:05.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:51:05.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:51:05.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:51:05.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:51:05.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:51:05.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:51:05.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:51:05.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:51:05.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:51:05.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:51:05.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:51:05.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:51:05.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f5a736f4-3482-41bf-9543-4ab259b1e387 2026-02-19T12:51:05.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 f5a736f4-3482-41bf-9543-4ab259b1e387' 2026-02-19T12:51:05.641 INFO:tasks.workunit.client.0.trial144.stdout:add osd3 f5a736f4-3482-41bf-9543-4ab259b1e387 2026-02-19T12:51:05.641 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:51:05.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA5B5dpzbS3JhAAxMn7bSUu5ji/+NJZwFEt9g== 2026-02-19T12:51:05.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA5B5dpzbS3JhAAxMn7bSUu5ji/+NJZwFEt9g=="}' 2026-02-19T12:51:05.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f5a736f4-3482-41bf-9543-4ab259b1e387 -i td/osd-scrub-repair/3/new.json 2026-02-19T12:51:05.811 INFO:tasks.workunit.client.0.trial144.stdout:3 2026-02-19T12:51:05.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-19T12:51:05.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA5B5dpzbS3JhAAxMn7bSUu5ji/+NJZwFEt9g== --osd-uuid f5a736f4-3482-41bf-9543-4ab259b1e387 2026-02-19T12:51:05.838 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:05.836+0000 7fd669ddca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:05.839 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:05.840+0000 7fd669ddca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:05.840 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:05.840+0000 7fd669ddca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:05.840 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:05.840+0000 7fd669ddca40 -1 bdev(0x55cb34a98c00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-19T12:51:05.840 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:05.840+0000 7fd669ddca40 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-19T12:51:06.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-19T12:51:06.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:51:06.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T12:51:06.444 INFO:tasks.workunit.client.0.trial144.stdout:adding osd3 key to auth repository 2026-02-19T12:51:06.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:51:06.650 INFO:tasks.workunit.client.0.trial144.stdout:start osd.3 2026-02-19T12:51:06.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T12:51:06.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:51:06.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:51:06.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:51:06.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:51:06.668 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:06.668+0000 7fb1554dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:06.669 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:06.668+0000 7fb1554dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:06.670 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:06.668+0000 7fb1554dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:06.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T12:51:06.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:51:06.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:51:06.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:51:06.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:51:06.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:06.811 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:51:06.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:51:06.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:06.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:51:06.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:51:07.072 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:07.072+0000 7fb1554dfa40 -1 Falling back to public interface 2026-02-19T12:51:07.703 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:07.704+0000 7fb1554dfa40 -1 osd.3 0 log_to_monitors true 2026-02-19T12:51:07.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:51:07.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:07.975 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:51:07.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:51:07.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:07.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:51:08.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:51:09.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:51:09.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:09.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:51:09.134 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:51:09.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:09.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:51:09.292 INFO:tasks.workunit.client.0.trial144.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1977871140,v1:127.0.0.1:6827/1977871140] [v2:127.0.0.1:6828/1977871140,v1:127.0.0.1:6829/1977871140] exists,up f5a736f4-3482-41bf-9543-4ab259b1e387 2026-02-19T12:51:09.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:51:09.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:51:09.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:51:09.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:866: corrupt_and_repair_jerasure: create_rbd_pool 2026-02-19T12:51:09.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T12:51:09.441 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T12:51:09.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T12:51:09.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T12:51:09.656 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T12:51:09.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:51:10.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T12:51:10.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:867: corrupt_and_repair_jerasure: wait_for_clean 2026-02-19T12:51:10.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:51:10.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:51:10.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:51:10.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:51:10.910 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:51:10.910 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:51:10.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:51:10.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:51:10.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:51:11.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:51:11.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:51:11.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:51:11.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:51:11.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:51:11.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:51:11.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:51:11.174 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:51:11.174 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:51:11.174 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T12:51:11.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:51:11.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:11.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:51:11.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-19T12:51:11.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-19T12:51:11.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-19T12:51:11.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:11.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:51:11.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672963 2026-02-19T12:51:11.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672963 2026-02-19T12:51:11.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-42949672963' 2026-02-19T12:51:11.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:11.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:51:11.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509443 2026-02-19T12:51:11.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509443 2026-02-19T12:51:11.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-42949672963 2-64424509443' 2026-02-19T12:51:11.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:11.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:51:11.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378626 2026-02-19T12:51:11.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378626 2026-02-19T12:51:11.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-42949672963 2-64424509443 3-81604378626' 2026-02-19T12:51:11.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:11.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-19T12:51:11.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:11.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:51:11.463 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-19T12:51:11.463 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:11.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-19T12:51:11.465 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836484 2026-02-19T12:51:11.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-19T12:51:11.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:11.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-19T12:51:11.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:51:12.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:51:12.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:12.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-19T12:51:12.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:51:13.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:51:13.794 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:13.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836484 2026-02-19T12:51:13.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:13.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672963 2026-02-19T12:51:13.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:13.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:51:13.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672963 2026-02-19T12:51:13.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:13.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672963 2026-02-19T12:51:13.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672963' 2026-02-19T12:51:13.958 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672963 2026-02-19T12:51:13.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:51:14.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672964 -lt 42949672963 2026-02-19T12:51:14.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:14.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509443 2026-02-19T12:51:14.120 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:14.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:51:14.121 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509443 2026-02-19T12:51:14.121 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:14.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509443 2026-02-19T12:51:14.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509443' 2026-02-19T12:51:14.123 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 64424509443 2026-02-19T12:51:14.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:51:14.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509443 -lt 64424509443 2026-02-19T12:51:14.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:14.284 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-81604378626 2026-02-19T12:51:14.284 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:14.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:51:14.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-81604378626 2026-02-19T12:51:14.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:14.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378626 2026-02-19T12:51:14.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 81604378626' 2026-02-19T12:51:14.288 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.3 seq 81604378626 2026-02-19T12:51:14.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:51:14.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378626 -lt 81604378626 2026-02-19T12:51:14.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:51:14.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:14.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:14.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:51:14.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:51:14.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:51:14.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:51:14.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:51:14.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:51:14.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:51:14.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:51:14.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:51:14.800 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:51:14.800 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:14.800 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:14.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:51:14.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:51:14.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:51:14.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:869: corrupt_and_repair_jerasure: create_ec_pool ecpool false k=2 m=2 2026-02-19T12:51:14.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: create_ec_pool: local pool_name=ecpool 2026-02-19T12:51:14.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: create_ec_pool: shift 2026-02-19T12:51:14.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: create_ec_pool: local allow_overwrites=false 2026-02-19T12:51:14.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2525: create_ec_pool: shift 2026-02-19T12:51:14.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=2 2026-02-19T12:51:15.200 INFO:tasks.workunit.client.0.trial144.stderr:load: isa 2026-02-19T12:51:15.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-19T12:51:15.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-19T12:51:15.507 INFO:tasks.workunit.client.0.trial144.stderr:pool 'ecpool' created 2026-02-19T12:51:15.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:51:16.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: create_ec_pool: '[' false = true ']' 2026-02-19T12:51:16.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2535: create_ec_pool: wait_for_clean 2026-02-19T12:51:16.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:51:16.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:51:16.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:51:16.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:51:16.521 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:51:16.521 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:51:16.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:51:16.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:51:16.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:51:16.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:51:16.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:51:16.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:51:16.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:51:16.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:51:16.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:51:16.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:51:16.792 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:51:16.793 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:51:16.793 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T12:51:16.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:51:16.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:16.793 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:51:16.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836486 2026-02-19T12:51:16.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836486 2026-02-19T12:51:16.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836486' 2026-02-19T12:51:16.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:16.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:51:16.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672966 2026-02-19T12:51:16.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672966 2026-02-19T12:51:16.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836486 1-42949672966' 2026-02-19T12:51:16.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:16.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:51:17.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509445 2026-02-19T12:51:17.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509445 2026-02-19T12:51:17.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836486 1-42949672966 2-64424509445' 2026-02-19T12:51:17.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:17.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:51:17.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378628 2026-02-19T12:51:17.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378628 2026-02-19T12:51:17.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836486 1-42949672966 2-64424509445 3-81604378628' 2026-02-19T12:51:17.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:17.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836486 2026-02-19T12:51:17.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:17.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:51:17.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836486 2026-02-19T12:51:17.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:17.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836486 2026-02-19T12:51:17.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-19T12:51:17.088 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836486 2026-02-19T12:51:17.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:17.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-19T12:51:17.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:17.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672966 2026-02-19T12:51:17.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:17.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:51:17.255 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672966 2026-02-19T12:51:17.255 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:17.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672966 2026-02-19T12:51:17.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-19T12:51:17.256 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672966 2026-02-19T12:51:17.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:51:17.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672966 -lt 42949672966 2026-02-19T12:51:17.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:17.420 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509445 2026-02-19T12:51:17.420 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:17.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:51:17.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509445 2026-02-19T12:51:17.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:17.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509445 2026-02-19T12:51:17.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509445' 2026-02-19T12:51:17.424 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 64424509445 2026-02-19T12:51:17.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:51:17.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509444 -lt 64424509445 2026-02-19T12:51:17.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:51:18.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:51:18.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:51:18.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509444 -lt 64424509445 2026-02-19T12:51:18.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:51:19.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:51:19.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:51:19.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509445 -lt 64424509445 2026-02-19T12:51:19.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:19.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-81604378628 2026-02-19T12:51:19.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:19.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:51:19.907 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-81604378628 2026-02-19T12:51:19.907 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:19.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378628 2026-02-19T12:51:19.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 81604378628' 2026-02-19T12:51:19.909 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.3 seq 81604378628 2026-02-19T12:51:19.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:51:20.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378629 -lt 81604378628 2026-02-19T12:51:20.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:51:20.080 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:20.080 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:20.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:51:20.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:51:20.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:51:20.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:51:20.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:51:20.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:51:20.277 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:51:20.277 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:51:20.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:51:20.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:51:20.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:20.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:20.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:51:20.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:51:20.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:51:20.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2536: create_ec_pool: return 0 2026-02-19T12:51:20.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:870: corrupt_and_repair_jerasure: corrupt_and_repair_erasure_coded td/osd-scrub-repair ecpool 2026-02-19T12:51:20.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:308: corrupt_and_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-19T12:51:20.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:309: corrupt_and_repair_erasure_coded: local poolname=ecpool 2026-02-19T12:51:20.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:311: corrupt_and_repair_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-19T12:51:20.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T12:51:20.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T12:51:20.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-19T12:51:20.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T12:51:20.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T12:51:20.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T12:51:20.832 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T12:51:20.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T12:51:21.037 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T12:51:21.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T12:51:21.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T12:51:21.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-19T12:51:21.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:313: corrupt_and_repair_erasure_coded: get_primary ecpool SOMETHING 2026-02-19T12:51:21.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=ecpool 2026-02-19T12:51:21.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:51:21.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:51:21.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:51:21.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:313: corrupt_and_repair_erasure_coded: local primary=3 2026-02-19T12:51:21.237 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: get_osds ecpool SOMETHING 2026-02-19T12:51:21.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T12:51:21.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: sed -e s/3// 2026-02-19T12:51:21.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T12:51:21.238 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:51:21.239 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T12:51:21.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T12:51:21.395 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:51:21.395 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:51:21.395 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T12:51:21.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 0 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: osds=('1' '2' '0') 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: local -a osds 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:315: corrupt_and_repair_erasure_coded: local not_primary_first=1 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:316: corrupt_and_repair_erasure_coded: local not_primary_second=2 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:319: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 3 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:286: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:287: corrupt_and_repair_one: local poolname=ecpool 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:288: corrupt_and_repair_one: local osd=3 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:293: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:51:21.396 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:51:21.397 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:51:21.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:51:21.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:51:21.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:51:21.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:51:21.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:51:21.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:51:21.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:51:21.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:51:21.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:51:21.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:51:21.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-19T12:51:21.866 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#2:eb822e21:::SOMETHING:head# 2026-02-19T12:51:21.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:51:21.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:51:21.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:51:21.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:51:21.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:51:21.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:51:21.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:51:21.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:51:21.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:51:21.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:51:21.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:51:21.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:51:21.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:51:21.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:51:21.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:51:21.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:51:21.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:51:21.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:51:21.980 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:51:21.980 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:51:21.980 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:51:21.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:21.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:21.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:51:21.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:51:21.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:51:21.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:51:21.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:51:21.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:51:21.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:51:21.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:51:21.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:51:21.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:51:21.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:51:21.982 INFO:tasks.workunit.client.0.trial144.stderr:start osd.3 2026-02-19T12:51:21.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:51:21.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:51:21.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:51:21.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:51:21.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:51:21.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:51:22.002 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:21.999+0000 7f954fb93a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:22.003 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:22.003+0000 7f954fb93a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:22.004 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:22.003+0000 7f954fb93a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:22.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:51:22.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:51:22.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:51:22.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:51:22.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:51:22.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:22.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:51:22.143 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:51:22.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:22.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:51:22.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:51:22.344 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:22.343+0000 7f954fb93a40 -1 Falling back to public interface 2026-02-19T12:51:22.973 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:22.971+0000 7f954fb93a40 -1 osd.3 33 log_to_monitors true 2026-02-19T12:51:23.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:51:23.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:23.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:51:23.303 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:51:23.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:23.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:51:23.457 INFO:tasks.workunit.client.0.trial144.stderr:osd.3 up in weight 1 up_from 37 up_thru 37 down_at 34 last_clean_interval [19,33) [v2:127.0.0.1:6826/2165075210,v1:127.0.0.1:6827/2165075210] [v2:127.0.0.1:6828/2165075210,v1:127.0.0.1:6829/2165075210] exists,up f5a736f4-3482-41bf-9543-4ab259b1e387 2026-02-19T12:51:23.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:51:23.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:51:23.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:51:23.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:51:23.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:51:23.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:51:23.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:51:23.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:51:23.459 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:51:23.459 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:51:23.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:51:23.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:51:23.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:51:23.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:51:23.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:51:23.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:51:23.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:51:23.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:51:23.573 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:51:23.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:51:23.729 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:51:23.729 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:51:23.729 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T12:51:23.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:51:23.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:23.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:51:23.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836489 2026-02-19T12:51:23.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836489 2026-02-19T12:51:23.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489' 2026-02-19T12:51:23.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:23.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:51:23.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672968 2026-02-19T12:51:23.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672968 2026-02-19T12:51:23.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672968' 2026-02-19T12:51:23.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:23.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:51:23.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509447 2026-02-19T12:51:23.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509447 2026-02-19T12:51:23.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672968 2-64424509447' 2026-02-19T12:51:23.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:23.946 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:51:24.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789954 2026-02-19T12:51:24.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789954 2026-02-19T12:51:24.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672968 2-64424509447 3-158913789954' 2026-02-19T12:51:24.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:24.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836489 2026-02-19T12:51:24.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:24.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:51:24.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836489 2026-02-19T12:51:24.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:24.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836489 2026-02-19T12:51:24.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-19T12:51:24.022 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836489 2026-02-19T12:51:24.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:24.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-19T12:51:24.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:51:25.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:51:25.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:25.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-19T12:51:25.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:25.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672968 2026-02-19T12:51:25.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:25.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:51:25.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672968 2026-02-19T12:51:25.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:25.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672968 2026-02-19T12:51:25.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2026-02-19T12:51:25.348 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 42949672968 2026-02-19T12:51:25.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:51:25.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672968 -lt 42949672968 2026-02-19T12:51:25.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:25.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509447 2026-02-19T12:51:25.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:25.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:51:25.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509447 2026-02-19T12:51:25.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:25.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509447 2026-02-19T12:51:25.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509447' 2026-02-19T12:51:25.510 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 64424509447 2026-02-19T12:51:25.511 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:51:25.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509447 -lt 64424509447 2026-02-19T12:51:25.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:25.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-158913789954 2026-02-19T12:51:25.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:25.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:51:25.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-158913789954 2026-02-19T12:51:25.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:25.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789954 2026-02-19T12:51:25.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 158913789954' 2026-02-19T12:51:25.673 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 158913789954 2026-02-19T12:51:25.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:51:25.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789954 -lt 158913789954 2026-02-19T12:51:25.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:51:25.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:25.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:26.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:51:26.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:51:26.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:51:26.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:51:26.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:51:26.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:51:26.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:51:26.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:51:26.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:51:26.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:51:26.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:26.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:26.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:51:26.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:51:26.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:51:26.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-19T12:51:26.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:51:26.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:51:26.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:51:26.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:51:26.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: local pg=2.0 2026-02-19T12:51:26.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: corrupt_and_repair_one: repair 2.0 2026-02-19T12:51:26.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:51:26.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:51:26.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:26.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:26.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:26.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:26.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:51:15.506267+0000 2026-02-19T12:51:26.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:51:26.839 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:51:26.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:51:15.506267+0000 2026-02-19T12:51:26.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:51:26.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:51:15.506267+0000 2026-02-19T12:51:26.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:51:26.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:51:26.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:51:26.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:51:26.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:26.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:26.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:26.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:27.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:51:15.506267+0000 '>' 2026-02-19T12:51:15.506267+0000 2026-02-19T12:51:27.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:51:28.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:51:28.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:51:28.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:51:28.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:28.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:28.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:28.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:28.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:51:15.506267+0000 '>' 2026-02-19T12:51:15.506267+0000 2026-02-19T12:51:28.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:51:29.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:51:29.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:51:29.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:51:29.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:29.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:29.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:29.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:29.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:51:27.997213+0000 '>' 2026-02-19T12:51:15.506267+0000 2026-02-19T12:51:29.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:51:29.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:302: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:51:29.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:51:29.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:51:29.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:51:29.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:51:29.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:51:29.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:51:29.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:51:29.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:51:29.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:51:29.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:51:29.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:51:29.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:51:29.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:51:29.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:51:29.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:51:29.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:51:29.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:51:29.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:51:29.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:51:29.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:51:29.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:51:29.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:51:29.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-19T12:51:29.702 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::deep_scrub_2.0s0:head#, (61) No data available 2026-02-19T12:51:29.702 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-19T12:51:29.702 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:51:29.702 INFO:tasks.workunit.client.0.trial144.stdout:hinfo_key 2026-02-19T12:51:29.702 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:51:29.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:51:29.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:51:29.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:51:29.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:51:29.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:51:29.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:51:29.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:51:29.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:51:29.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:51:29.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:51:29.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:51:29.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:51:29.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:51:29.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:51:29.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:51:29.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:51:29.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:51:29.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:51:29.797 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:51:29.798 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:51:29.798 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:51:29.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:29.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:29.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:51:29.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:51:29.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:51:29.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:51:29.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:51:29.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:51:29.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:51:29.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:51:29.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:51:29.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:51:29.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:51:29.799 INFO:tasks.workunit.client.0.trial144.stderr:start osd.3 2026-02-19T12:51:29.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:51:29.800 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:51:29.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:51:29.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:51:29.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:51:29.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:51:29.816 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:29.815+0000 7f4bc9fc8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:29.817 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:29.815+0000 7f4bc9fc8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:29.818 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:29.815+0000 7f4bc9fc8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:29.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:51:29.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:51:29.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:51:29.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:51:29.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:51:29.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:29.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:51:29.962 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:51:29.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:29.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:51:30.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:51:30.240 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:30.239+0000 7f4bc9fc8a40 -1 Falling back to public interface 2026-02-19T12:51:30.821 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:30.819+0000 7f4bc9fc8a40 -1 osd.3 39 log_to_monitors true 2026-02-19T12:51:31.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:51:31.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:31.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:51:31.121 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:51:31.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:31.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:51:31.275 INFO:tasks.workunit.client.0.trial144.stderr:osd.3 up in weight 1 up_from 43 up_thru 43 down_at 40 last_clean_interval [37,39) [v2:127.0.0.1:6826/1586696034,v1:127.0.0.1:6827/1586696034] [v2:127.0.0.1:6828/1586696034,v1:127.0.0.1:6829/1586696034] exists,up f5a736f4-3482-41bf-9543-4ab259b1e387 2026-02-19T12:51:31.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:51:31.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:51:31.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:51:31.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:51:31.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:51:31.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:51:31.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:51:31.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:51:31.276 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:51:31.276 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:51:31.277 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:51:31.277 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:51:31.277 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:51:31.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:51:31.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:51:31.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:51:31.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:51:31.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:51:31.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:51:31.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:51:31.552 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:51:31.552 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:51:31.552 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T12:51:31.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:51:31.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:31.553 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:51:31.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-19T12:51:31.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-19T12:51:31.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491' 2026-02-19T12:51:31.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:31.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:51:31.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672970 2026-02-19T12:51:31.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672970 2026-02-19T12:51:31.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672970' 2026-02-19T12:51:31.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:31.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:51:31.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509450 2026-02-19T12:51:31.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509450 2026-02-19T12:51:31.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509450' 2026-02-19T12:51:31.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:31.770 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:51:31.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593730 2026-02-19T12:51:31.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593730 2026-02-19T12:51:31.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509450 3-184683593730' 2026-02-19T12:51:31.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:31.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-19T12:51:31.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:31.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:51:31.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836491 2026-02-19T12:51:31.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:31.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836491 2026-02-19T12:51:31.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-19T12:51:31.848 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836491 2026-02-19T12:51:31.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:32.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T12:51:32.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:51:33.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:51:33.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:33.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836492 -lt 21474836491 2026-02-19T12:51:33.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:33.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672970 2026-02-19T12:51:33.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:33.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:51:33.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672970 2026-02-19T12:51:33.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:33.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672970 2026-02-19T12:51:33.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-19T12:51:33.168 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 42949672970 2026-02-19T12:51:33.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:51:33.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672971 -lt 42949672970 2026-02-19T12:51:33.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:33.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509450 2026-02-19T12:51:33.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:33.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:51:33.335 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509450 2026-02-19T12:51:33.335 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:33.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509450 2026-02-19T12:51:33.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509450' 2026-02-19T12:51:33.337 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 64424509450 2026-02-19T12:51:33.337 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:51:33.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509450 -lt 64424509450 2026-02-19T12:51:33.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:33.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-184683593730 2026-02-19T12:51:33.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:33.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:51:33.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-184683593730 2026-02-19T12:51:33.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:33.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593730 2026-02-19T12:51:33.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 184683593730' 2026-02-19T12:51:33.508 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 184683593730 2026-02-19T12:51:33.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:51:33.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593730 -lt 184683593730 2026-02-19T12:51:33.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:51:33.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:33.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:33.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:51:33.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:51:33.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:51:33.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:51:33.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:51:33.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:51:33.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:51:33.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:51:34.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:51:34.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:51:34.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:34.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:34.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:51:34.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:51:34.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:51:34.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:51:34.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:304: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:51:34.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:321: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 1 2026-02-19T12:51:34.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:286: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-19T12:51:34.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:287: corrupt_and_repair_one: local poolname=ecpool 2026-02-19T12:51:34.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:288: corrupt_and_repair_one: local osd=1 2026-02-19T12:51:34.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:293: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:51:34.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:51:34.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:51:34.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:51:34.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:51:34.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:51:34.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:51:34.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:51:34.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:51:34.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:51:34.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:51:34.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:51:34.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:51:34.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:51:34.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:51:34.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:51:34.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:51:34.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:51:34.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:51:34.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:51:34.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:51:34.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:51:34.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:51:34.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-19T12:51:34.776 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-19T12:51:34.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:51:34.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:51:34.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:51:34.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:51:34.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:51:34.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:51:34.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:51:34.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:51:34.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:51:34.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:51:34.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:51:34.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:51:34.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:51:34.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:51:34.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:51:34.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:51:34.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:51:34.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:51:34.875 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:51:34.876 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:51:34.876 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:51:34.876 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:34.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:34.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:51:34.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:51:34.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:51:34.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:51:34.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:51:34.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:51:34.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:51:34.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:51:34.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:51:34.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:51:34.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:51:34.878 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:51:34.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:51:34.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:51:34.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:51:34.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:51:34.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:51:34.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:51:34.892 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:34.891+0000 7fce4c7c4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:34.894 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:34.891+0000 7fce4c7c4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:34.895 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:34.895+0000 7fce4c7c4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:35.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:51:35.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:51:35.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:51:35.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:51:35.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:51:35.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:35.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:51:35.036 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:51:35.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:35.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:51:35.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:51:35.332 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:35.331+0000 7fce4c7c4a40 -1 Falling back to public interface 2026-02-19T12:51:35.982 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:35.979+0000 7fce4c7c4a40 -1 osd.1 45 log_to_monitors true 2026-02-19T12:51:36.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:51:36.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:36.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:51:36.195 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:51:36.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:36.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:51:36.354 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 49 up_thru 49 down_at 46 last_clean_interval [10,45) [v2:127.0.0.1:6810/836333278,v1:127.0.0.1:6811/836333278] [v2:127.0.0.1:6812/836333278,v1:127.0.0.1:6813/836333278] exists,up 6a06dd6f-2a04-4584-96dd-5506f512ce78 2026-02-19T12:51:36.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:51:36.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:51:36.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:51:36.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:51:36.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:51:36.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:51:36.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:51:36.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:51:36.356 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:51:36.356 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:51:36.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:51:36.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:51:36.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:51:36.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:51:36.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:51:36.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:51:36.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:51:36.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:51:36.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:51:36.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:51:36.628 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:51:36.628 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:51:36.628 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T12:51:36.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:51:36.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:36.628 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:51:36.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836493 2026-02-19T12:51:36.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836493 2026-02-19T12:51:36.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493' 2026-02-19T12:51:36.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:36.702 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:51:36.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397506 2026-02-19T12:51:36.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397506 2026-02-19T12:51:36.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-210453397506' 2026-02-19T12:51:36.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:36.774 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:51:36.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509452 2026-02-19T12:51:36.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509452 2026-02-19T12:51:36.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-210453397506 2-64424509452' 2026-02-19T12:51:36.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:36.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:51:36.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593732 2026-02-19T12:51:36.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593732 2026-02-19T12:51:36.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-210453397506 2-64424509452 3-184683593732' 2026-02-19T12:51:36.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:36.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836493 2026-02-19T12:51:36.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:36.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:51:36.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836493 2026-02-19T12:51:36.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:36.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836493 2026-02-19T12:51:36.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-19T12:51:36.928 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836493 2026-02-19T12:51:36.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:37.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836493 -lt 21474836493 2026-02-19T12:51:37.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:37.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-210453397506 2026-02-19T12:51:37.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:37.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:51:37.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-210453397506 2026-02-19T12:51:37.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:37.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397506 2026-02-19T12:51:37.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 210453397506' 2026-02-19T12:51:37.090 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 210453397506 2026-02-19T12:51:37.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:51:37.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397506 -lt 210453397506 2026-02-19T12:51:37.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:37.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509452 2026-02-19T12:51:37.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:37.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:51:37.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509452 2026-02-19T12:51:37.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:37.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509452 2026-02-19T12:51:37.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509452' 2026-02-19T12:51:37.262 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 64424509452 2026-02-19T12:51:37.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:51:37.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509452 -lt 64424509452 2026-02-19T12:51:37.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:37.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-184683593732 2026-02-19T12:51:37.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:37.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:51:37.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-184683593732 2026-02-19T12:51:37.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:37.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593732 2026-02-19T12:51:37.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 184683593732' 2026-02-19T12:51:37.440 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 184683593732 2026-02-19T12:51:37.440 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:51:37.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593732 -lt 184683593732 2026-02-19T12:51:37.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:51:37.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:37.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:37.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:51:37.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:51:37.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:51:37.793 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:51:37.793 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:51:37.793 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:51:37.793 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:51:37.793 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:51:37.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:51:37.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:51:37.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:37.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:38.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:51:38.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:51:38.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:51:38.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-19T12:51:38.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:51:38.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:51:38.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:51:38.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:51:38.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: local pg=2.0 2026-02-19T12:51:38.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: corrupt_and_repair_one: repair 2.0 2026-02-19T12:51:38.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:51:38.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:51:38.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:38.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:38.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:38.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:38.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:51:27.997213+0000 2026-02-19T12:51:38.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:51:38.601 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:51:38.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:51:27.997213+0000 2026-02-19T12:51:38.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:51:38.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:51:27.997213+0000 2026-02-19T12:51:38.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:51:38.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:51:38.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:51:38.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:51:38.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:38.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:38.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:38.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:38.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:51:27.997213+0000 '>' 2026-02-19T12:51:27.997213+0000 2026-02-19T12:51:38.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:51:39.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:51:39.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:51:39.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:51:39.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:39.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:39.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:39.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:39.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:51:27.997213+0000 '>' 2026-02-19T12:51:27.997213+0000 2026-02-19T12:51:39.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:51:40.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:51:40.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:51:40.932 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:51:40.932 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:40.932 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:40.932 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:40.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:41.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:51:38.906705+0000 '>' 2026-02-19T12:51:27.997213+0000 2026-02-19T12:51:41.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:51:41.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:302: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:51:41.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:51:41.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:51:41.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:51:41.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:51:41.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:51:41.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:51:41.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:51:41.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:51:41.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:51:41.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:51:41.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:51:41.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:51:41.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:51:41.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:51:41.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:51:41.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:51:41.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:51:41.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:51:41.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:51:41.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:51:41.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:51:41.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:51:41.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-19T12:51:41.481 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::deep_scrub_2.0s0:head#, (61) No data available 2026-02-19T12:51:41.482 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-19T12:51:41.482 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:51:41.482 INFO:tasks.workunit.client.0.trial144.stdout:hinfo_key 2026-02-19T12:51:41.482 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:51:41.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:51:41.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:51:41.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:51:41.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:51:41.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:51:41.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:51:41.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:51:41.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:51:41.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:51:41.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:51:41.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:51:41.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:51:41.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:51:41.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:51:41.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:51:41.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:51:41.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:51:41.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:51:41.581 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:51:41.581 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:51:41.581 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:51:41.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:41.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:41.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:51:41.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:51:41.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:51:41.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:51:41.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:51:41.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:51:41.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:51:41.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:51:41.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:51:41.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:51:41.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:51:41.584 INFO:tasks.workunit.client.0.trial144.stderr:start osd.3 2026-02-19T12:51:41.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:51:41.584 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:51:41.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:51:41.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:51:41.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:51:41.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:51:41.591 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:41.591+0000 7fd8521b6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:41.593 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:41.591+0000 7fd8521b6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:41.593 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:41.591+0000 7fd8521b6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:41.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:51:41.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:51:41.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:51:41.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:51:41.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:51:41.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:41.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:51:41.735 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:51:41.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:41.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:51:41.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:51:42.016 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:42.015+0000 7fd8521b6a40 -1 Falling back to public interface 2026-02-19T12:51:42.693 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:51:42.691+0000 7fd8521b6a40 -1 osd.3 50 log_to_monitors true 2026-02-19T12:51:42.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:51:42.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:42.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:51:42.892 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:51:42.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:42.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:51:43.053 INFO:tasks.workunit.client.0.trial144.stderr:osd.3 up in weight 1 up_from 54 up_thru 54 down_at 51 last_clean_interval [43,50) [v2:127.0.0.1:6826/1067726266,v1:127.0.0.1:6827/1067726266] [v2:127.0.0.1:6828/1067726266,v1:127.0.0.1:6829/1067726266] exists,up f5a736f4-3482-41bf-9543-4ab259b1e387 2026-02-19T12:51:43.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:51:43.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:51:43.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:51:43.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:51:43.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:51:43.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:51:43.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:51:43.054 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:51:43.055 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:51:43.055 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:51:43.055 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:51:43.055 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:51:43.055 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:51:43.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:51:43.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:51:43.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:51:43.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:51:43.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:51:43.156 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:51:43.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:51:43.319 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:51:43.319 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:51:43.320 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T12:51:43.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:51:43.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:43.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:51:43.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836496 2026-02-19T12:51:43.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836496 2026-02-19T12:51:43.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836496' 2026-02-19T12:51:43.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:43.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:51:43.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397508 2026-02-19T12:51:43.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397508 2026-02-19T12:51:43.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836496 1-210453397508' 2026-02-19T12:51:43.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:43.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:51:43.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509454 2026-02-19T12:51:43.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509454 2026-02-19T12:51:43.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836496 1-210453397508 2-64424509454' 2026-02-19T12:51:43.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:43.538 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:51:43.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=231928233986 2026-02-19T12:51:43.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 231928233986 2026-02-19T12:51:43.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836496 1-210453397508 2-64424509454 3-231928233986' 2026-02-19T12:51:43.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:43.609 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836496 2026-02-19T12:51:43.609 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:43.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:51:43.611 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836496 2026-02-19T12:51:43.611 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:43.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836496 2026-02-19T12:51:43.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836496' 2026-02-19T12:51:43.613 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836496 2026-02-19T12:51:43.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:43.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836495 -lt 21474836496 2026-02-19T12:51:43.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:51:44.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:51:44.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:44.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836495 -lt 21474836496 2026-02-19T12:51:44.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:51:45.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:51:45.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:46.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836496 -lt 21474836496 2026-02-19T12:51:46.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:46.101 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-210453397508 2026-02-19T12:51:46.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:46.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:51:46.104 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-210453397508 2026-02-19T12:51:46.104 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:46.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397508 2026-02-19T12:51:46.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 210453397508' 2026-02-19T12:51:46.106 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 210453397508 2026-02-19T12:51:46.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:51:46.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397508 -lt 210453397508 2026-02-19T12:51:46.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:46.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509454 2026-02-19T12:51:46.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:46.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:51:46.274 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509454 2026-02-19T12:51:46.274 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:46.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509454 2026-02-19T12:51:46.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509454' 2026-02-19T12:51:46.276 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 64424509454 2026-02-19T12:51:46.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:51:46.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509454 -lt 64424509454 2026-02-19T12:51:46.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:46.437 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-231928233986 2026-02-19T12:51:46.437 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:46.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:51:46.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-231928233986 2026-02-19T12:51:46.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:46.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=231928233986 2026-02-19T12:51:46.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 231928233986' 2026-02-19T12:51:46.441 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 231928233986 2026-02-19T12:51:46.441 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:51:46.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 231928233986 -lt 231928233986 2026-02-19T12:51:46.610 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:51:46.610 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:46.610 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:46.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:51:46.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:51:46.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:51:46.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:51:46.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:51:46.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:51:46.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:51:46.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:51:46.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:51:46.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:51:46.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:46.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:47.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:51:47.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:51:47.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:51:47.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:51:47.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:304: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:51:47.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:322: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 1 2 2026-02-19T12:51:47.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:245: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-19T12:51:47.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:246: corrupt_and_repair_two: local poolname=ecpool 2026-02-19T12:51:47.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:247: corrupt_and_repair_two: local first=1 2026-02-19T12:51:47.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_two: local second=2 2026-02-19T12:51:47.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_two: pids= 2026-02-19T12:51:47.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:51:47.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:51:47.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:51:47.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 128317"' 2026-02-19T12:51:47.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 128317' 2026-02-19T12:51:47.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T12:51:47.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:51:47.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:51:47.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 128319"' 2026-02-19T12:51:47.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 128319' 2026-02-19T12:51:47.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_two: wait_background pids 2026-02-19T12:51:47.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 128317 128319' 2026-02-19T12:51:47.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:51:47.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:51:47.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 128317 2026-02-19T12:51:47.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:51:47.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/128320: /' 2026-02-19T12:51:47.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T12:51:47.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/128322: /' 2026-02-19T12:51:47.835 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:51:47.835 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:51:47.835 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T12:51:47.835 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:51:47.835 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T12:51:47.835 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:51:47.835 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:51:47.835 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T12:51:47.835 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:51:47.835 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-19T12:51:47.835 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:51:47.835 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:51:47.835 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:51:47.835 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING remove 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: remove 2#2:eb822e21:::SOMETHING:head# 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:51:47.836 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: 128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:51:47.837 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:51:47.838 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:51:47.838 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:51:47.838 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:51:47.838 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:51:47.838 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:51:47.838 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:51:47.838 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:51:47.839 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128322: start osd.2 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:1activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128320: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:51:47.840 INFO:tasks.workunit.client.0.trial144.stderr:128320: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: start osd.1 2026-02-19T12:51:47.841 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:51:49.319 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-19T12:51:49.319 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T12:51:49.319 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:51:49.319 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:51:49.319 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:51:49.319 INFO:tasks.workunit.client.0.trial144.stderr:128322: 2026-02-19T12:51:47.851+0000 7f9cd9626a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:49.319 INFO:tasks.workunit.client.0.trial144.stderr:128322: 2026-02-19T12:51:47.855+0000 7f9cd9626a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:49.319 INFO:tasks.workunit.client.0.trial144.stderr:128322: 2026-02-19T12:51:47.855+0000 7f9cd9626a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:49.319 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: 0 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: 2026-02-19T12:51:48.339+0000 7f9cd9626a40 -1 Falling back to public interface 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: 2026-02-19T12:51:48.987+0000 7f9cd9626a40 -1 osd.2 55 log_to_monitors true 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: 1 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: osd.2 up in weight 1 up_from 59 up_thru 46 down_at 56 last_clean_interval [15,55) [v2:127.0.0.1:6818/731201308,v1:127.0.0.1:6819/731201308] [v2:127.0.0.1:6820/731201308,v1:127.0.0.1:6821/731201308] exists,up 56b9b2cb-98f6-4b89-81c3-a3d557f1da96 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:51:49.320 INFO:tasks.workunit.client.0.trial144.stderr:128322: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:51:49.321 INFO:tasks.workunit.client.0.trial144.stderr:128322: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_t28320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:51:49.321 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:51:49.321 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:51:49.321 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: 2026-02-19T12:51:47.851+0000 7f0a41fcea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: 2026-02-19T12:51:47.855+0000 7f0a41fcea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: 2026-02-19T12:51:47.855+0000 7f0a41fcea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: 0 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: 2026-02-19T12:51:48.331+0000 7f0a41fcea40 -1 Falling back to public interface 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: 2026-02-19T12:51:48.963+0000 7f0a41fcea40 -1 osd.1 55 log_to_monitors true 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: 1 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: osd.1 up in weight 1 up_from 59 up_thru 59 down_at 56 last_clean_interval [49,55) [v2:127.0.0.1:6810/2748313661,v1:127.0.0.1:6811/2748313661] [v2:127.0.0.1:6812/2748313661,v1:127.0.0.1:6813/2748313661] exists,up 6a06dd6f-2a04-4584-96dd-5506f512ce78 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:51:49.322 INFO:tasks.workunit.client.0.trial144.stderr:128320: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:51:49.873 INFO:tasks.workunit.client.0.trial144.stderr:128320: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: gimeout_delays: echo true 2026-02-19T12:51:49.873 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:51:49.873 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:51:49.873 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:51:49.873 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:51:49.873 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:51:49.873 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:51:49.874 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:51:49.874 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:51:49.874 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:51:49.874 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:51:49.874 INFO:tasks.workunit.client.0.trial144.stderr:128322: 1 2026-02-19T12:51:49.874 INFO:tasks.workunit.client.0.trial144.stderr:128322: 2 2026-02-19T12:51:49.874 INFO:tasks.workunit.client.0.trial144.stderr:128322: 3' 2026-02-19T12:51:49.874 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:51:49.874 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:49.874 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:51:49.874 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836498 2026-02-19T12:51:49.875 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836498 2026-02-19T12:51:49.875 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836498' 2026-02-19T12:51:49.875 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:49.875 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:51:49.875 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070466 2026-02-19T12:51:49.875 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070466 2026-02-19T12:51:49.875 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836498 1-253403070466' 2026-02-19T12:51:49.875 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:49.875 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:51:49.875 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070466 2026-02-19T12:51:49.875 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070466 2026-02-19T12:51:49.875 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836498 1-253403070466 2-253403070466' 2026-02-19T12:51:49.876 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:49.876 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:51:49.876 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=231928233988 2026-02-19T12:51:49.876 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 231928233988 2026-02-19T12:51:49.876 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836498 1-253403070466 2-253403070466 3-231928233988' 2026-02-19T12:51:49.876 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:49.876 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836498 2026-02-19T12:51:49.877 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubunet_timeout_delays: echo true 2026-02-19T12:51:49.877 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:51:49.877 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:51:49.877 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:51:49.877 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:51:49.877 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:51:49.877 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:51:49.877 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:51:49.877 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:51:49.877 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:51:49.878 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:51:49.878 INFO:tasks.workunit.client.0.trial144.stderr:128320: 1 2026-02-19T12:51:49.878 INFO:tasks.workunit.client.0.trial144.stderr:128320: 2 2026-02-19T12:51:49.878 INFO:tasks.workunit.client.0.trial144.stderr:128320: 3' 2026-02-19T12:51:49.878 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:51:49.878 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:49.878 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:51:49.878 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836499 2026-02-19T12:51:49.878 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836499 2026-02-19T12:51:49.878 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499' 2026-02-19T12:51:49.878 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:49.878 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:51:49.879 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070467 2026-02-19T12:51:49.879 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070467 2026-02-19T12:51:49.879 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-253403070467' 2026-02-19T12:51:49.879 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:49.879 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:51:49.879 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070467 2026-02-19T12:51:49.879 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070467 2026-02-19T12:51:49.879 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-253403070467 2-253403070467' 2026-02-19T12:51:49.879 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:51:49.879 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:51:49.879 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=231928233989 2026-02-19T12:51:49.879 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 231928233989 2026-02-19T12:51:49.880 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-253403070467 2-253403070467 3-231928233989' 2026-02-19T12:51:49.880 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:49.880 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836499 2026-02-19T12:51:51.534 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/tu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:51.534 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:51:51.534 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836498 2026-02-19T12:51:51.535 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:51.535 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836498 2026-02-19T12:51:51.535 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2026-02-19T12:51:51.535 INFO:tasks.workunit.client.0.trial144.stderr:128322: waiting osd.0 seq 21474836498 2026-02-19T12:51:51.535 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:51.535 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836497 -lt 21474836498 2026-02-19T12:51:51.535 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:51:51.535 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:51:51.535 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:51.535 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836499 -lt 21474836498 2026-02-19T12:51:51.535 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:51.536 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-253403070466 2026-02-19T12:51:51.536 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:51.536 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:51:51.536 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-253403070466 2026-02-19T12:51:51.536 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:51.536 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=253403070466 2026-02-19T12:51:51.536 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 253403070466' 2026-02-19T12:51:51.536 INFO:tasks.workunit.client.0.trial144.stderr:128322: waiting osd.1 seq 253403070466 2026-02-19T12:51:51.536 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:51:51.536 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 253403070467 -lt 253403070466 2026-02-19T12:51:51.536 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:51.536 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-253403070466 2026-02-19T12:51:51.537 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:51.537 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:51:51.537 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-253403070466 2026-02-19T12:51:51.537 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:51.537 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=253403070466 2026-02-19T12:51:51.537 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 253403070466' 2026-02-19T12:51:51.537 INFO:tasks.workunit.client.0.trial144.stderr:128322: waiting osd.2 seq 253403070466 2026-02-19T12:51:51.537 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:51:51.537 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 253403070467 -lt 253403070466 2026-02-19T12:51:51.537 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:51.539 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:226ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:51.539 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:51:51.539 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836499 2026-02-19T12:51:51.539 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:51.539 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836499 2026-02-19T12:51:51.539 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2026-02-19T12:51:51.540 INFO:tasks.workunit.client.0.trial144.stderr:128320: waiting osd.0 seq 21474836499 2026-02-19T12:51:51.540 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:51.540 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836497 -lt 21474836499 2026-02-19T12:51:51.540 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:51:51.540 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:51:51.540 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:51:51.540 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836499 -lt 21474836499 2026-02-19T12:51:51.540 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:51.540 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-253403070467 2026-02-19T12:51:51.540 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:51.540 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:51:51.540 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-253403070467 2026-02-19T12:51:51.541 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:51.541 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=253403070467 2026-02-19T12:51:51.541 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 253403070467' 2026-02-19T12:51:51.541 INFO:tasks.workunit.client.0.trial144.stderr:128320: waiting osd.1 seq 253403070467 2026-02-19T12:51:51.541 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:51:51.541 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 253403070467 -lt 253403070467 2026-02-19T12:51:51.541 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:51.541 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-253403070467 2026-02-19T12:51:51.541 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:51.541 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:51:51.541 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-253403070467 2026-02-19T12:51:51.541 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:51.542 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=253403070467 2026-02-19T12:51:51.542 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 253403070467' 2026-02-19T12:51:51.542 INFO:tasks.workunit.client.0.trial144.stderr:128320: waiting osd.2 seq 253403070467 2026-02-19T12:51:51.542 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:51:51.542 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 253403070467 -lt 253403070467 2026-02-19T12:51:51.542 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:51:52.238 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh7: flush_pg_stats: echo 3-231928233988 2026-02-19T12:51:52.239 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:52.239 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:51:52.239 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-231928233988 2026-02-19T12:51:52.239 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:52.239 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=231928233988 2026-02-19T12:51:52.239 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 231928233988' 2026-02-19T12:51:52.239 INFO:tasks.workunit.client.0.trial144.stderr:128322: waiting osd.3 seq 231928233988 2026-02-19T12:51:52.239 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:51:52.239 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 231928233989 -lt 231928233988 2026-02-19T12:51:52.239 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:51:52.240 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:52.240 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:52.240 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:51:52.240 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:51:52.240 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:51:52.240 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:51:52.240 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:51:52.240 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:51:52.240 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:51:52.240 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:51:52.240 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:51:52.240 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:51:52.240 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:52.241 INFO:tasks.workunit.client.0.trial144.stderr:128322: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:52.241 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:51:52.241 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:51:52.241 INFO:tasks.workunit.client.0.trial144.stderr:128322: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:51:52.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:51:52.260 INFO:tasks.workunit.client.0.trial144.stderr::2267: flush_pg_stats: echo 3-231928233989 2026-02-19T12:51:52.261 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:51:52.261 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:51:52.261 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-231928233989 2026-02-19T12:51:52.261 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:51:52.261 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=231928233989 2026-02-19T12:51:52.261 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 231928233989' 2026-02-19T12:51:52.261 INFO:tasks.workunit.client.0.trial144.stderr:128320: waiting osd.3 seq 231928233989 2026-02-19T12:51:52.261 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:51:52.261 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 231928233989 -lt 231928233989 2026-02-19T12:51:52.261 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:51:52.261 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:52.262 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:52.262 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:51:52.262 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:51:52.262 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:51:52.262 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:51:52.262 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:51:52.262 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:51:52.262 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:51:52.262 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:51:52.262 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:51:52.262 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:51:52.262 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:51:52.263 INFO:tasks.workunit.client.0.trial144.stderr:128320: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:51:52.263 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:51:52.263 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:51:52.263 INFO:tasks.workunit.client.0.trial144.stderr:128320: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:51:52.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:51:52.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:51:52.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 128319 2026-02-19T12:51:52.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:51:52.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:51:52.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:51:52.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:257: corrupt_and_repair_two: return_code=0 2026-02-19T12:51:52.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:258: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:51:52.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-19T12:51:52.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:51:52.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:51:52.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:51:52.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:51:52.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: local pg=2.0 2026-02-19T12:51:52.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:264: corrupt_and_repair_two: repair 2.0 2026-02-19T12:51:52.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:51:52.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:51:52.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:52.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:52.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:52.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:52.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:51:38.906705+0000 2026-02-19T12:51:52.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:51:52.735 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:51:52.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:51:38.906705+0000 2026-02-19T12:51:52.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:51:52.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:51:38.906705+0000 2026-02-19T12:51:52.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:51:52.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:51:52.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:51:52.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:51:52.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:52.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:52.748 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:52.748 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:52.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:51:38.906705+0000 '>' 2026-02-19T12:51:38.906705+0000 2026-02-19T12:51:52.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:51:53.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:51:53.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:51:53.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:51:53.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:53.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:53.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:53.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:54.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:51:38.906705+0000 '>' 2026-02-19T12:51:38.906705+0000 2026-02-19T12:51:54.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:51:55.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:51:55.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:51:55.070 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:51:55.070 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:55.070 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:55.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:55.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:55.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:51:38.906705+0000 '>' 2026-02-19T12:51:38.906705+0000 2026-02-19T12:51:55.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:51:56.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:51:56.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:51:56.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:51:56.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:56.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:56.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:56.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:56.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:51:38.906705+0000 '>' 2026-02-19T12:51:38.906705+0000 2026-02-19T12:51:56.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:51:57.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:51:57.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:51:57.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:51:57.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:57.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:57.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:57.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:57.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:51:38.906705+0000 '>' 2026-02-19T12:51:38.906705+0000 2026-02-19T12:51:57.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:51:58.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:51:58.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:51:58.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:51:58.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:58.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:58.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:58.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:58.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:51:38.906705+0000 '>' 2026-02-19T12:51:38.906705+0000 2026-02-19T12:51:58.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:51:59.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:51:59.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:51:59.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:51:59.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:51:59.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:51:59.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:51:59.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:51:59.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:51:53.611369+0000 '>' 2026-02-19T12:51:38.906705+0000 2026-02-19T12:51:59.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:51:59.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:268: corrupt_and_repair_two: pids= 2026-02-19T12:51:59.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:269: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:51:59.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:51:59.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:51:59.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 131179"' 2026-02-19T12:51:59.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 131179' 2026-02-19T12:51:59.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:270: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-19T12:51:59.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:51:59.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:51:59.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 131180"' 2026-02-19T12:51:59.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 131180' 2026-02-19T12:51:59.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:271: corrupt_and_repair_two: wait_background pids 2026-02-19T12:51:59.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 131179 131180' 2026-02-19T12:51:59.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:51:59.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:51:59.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 131179 2026-02-19T12:51:59.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:51:59.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/131182: /' 2026-02-19T12:51:59.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-19T12:51:59.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/131184: /' 2026-02-19T12:52:00.402 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:52:00.403 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:52:00.403 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T12:52:00.403 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:52:00.403 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-19T12:52:00.403 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:52:00.403 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:52:00.403 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T12:52:00.403 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:52:00.403 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-19T12:52:00.403 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:52:00.403 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:52:00.404 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:52:00.404 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:52:00.404 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:52:00.404 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:52:00.404 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-19T12:52:00.404 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:52:00.404 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:52:00.404 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T12:52:00.404 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:52:00.404 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:52:00.404 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING list-attrs 2026-02-19T12:52:00.404 INFO:tasks.workunit.client.0.trial144.stderr:131184: _ 2026-02-19T12:52:00.404 INFO:tasks.workunit.client.0.trial144.stderr:131184: hinfo_key 2026-02-19T12:52:00.405 INFO:tasks.workunit.client.0.trial144.stderr:131184: snapset 2026-02-19T12:52:00.405 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-19T12:52:00.405 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:52:00.405 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:52:00.405 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T12:52:00.405 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:52:00.405 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:52:00.405 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:52:00.405 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:52:00.406 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:52:00.406 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:52:00.406 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:52:00.406 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:52:00.406 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:52:00.406 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:52:00.406 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:52:00.407 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:52:00.407 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:52:00.407 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:52:00.407 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:52:00.407 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:52:00.407 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:52:00.407 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:52:00.407 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:52:00.407 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:52:00.407 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:52:00.407 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:52:00.407 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:52:00.408 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:52:00.408 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:52:00.408 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:52:00.408 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-19T12:52:00.408 INFO:tasks.workunit.client.0.trial144.stderr:131182: _ 2026-02-19T12:52:00.408 INFO:tasks.workunit.client.0.trial144.stderr:131182: hinfo_key 2026-02-19T12:52:00.408 INFO:tasks.workunit.client.0.trial144.stderr:131182: snapset 2026-02-19T12:52:00.408 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:52:00.408 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:52:00.408 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:52:00.408 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:52:00.408 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:52:00.409 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:52:00.409 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:52:00.409 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:52:00.409 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:52:00.409 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:52:00.409 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:52:00.409 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:52:00.409 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:52:00.409 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:52:00.409 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:52:00.409 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:52:00.409 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:52:00.409 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:52:00.410 INFO:tasks.workunit.client.0.trial144.stderr:131184: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:52:00.410 INFO:tasks.workunit.client.0.trial144.stderr:131184: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:00.410 INFO:tasks.workunit.client.0.trial144.stderr:131184: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:00.410 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:00.410 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:00.410 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:52:00.410 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:52:00.410 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:52:00.410 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:52:00.410 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:52:00.410 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:52:00.410 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:52:00.410 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:52:00.411 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:52:00.411 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:52:00.411 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T12:52:00.411 INFO:tasks.workunit.client.0.trial144.stderr:131184: start osd.2 2026-02-19T12:52:00.411 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recos.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:52:00.411 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:52:00.411 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:52:00.412 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:52:00.412 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:52:00.412 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:52:00.412 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:52:00.412 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:52:00.412 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:52:00.412 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:52:00.412 INFO:tasks.workunit.client.0.trial144.stderr:131182: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:52:00.412 INFO:tasks.workunit.client.0.trial144.stderr:131182: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:00.412 INFO:tasks.workunit.client.0.trial144.stderr:131182: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:00.412 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:00.412 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:00.413 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:52:00.413 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:52:00.413 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:52:00.413 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:52:00.413 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:52:00.413 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:52:00.413 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:52:00.413 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:52:00.413 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:52:00.413 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:52:00.413 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:52:00.413 INFO:tasks.workunit.client.0.trial144.stderr:131182: start osd.1 2026-02-19T12:52:01.889 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:52:01.889 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-19T12:52:01.890 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T12:52:01.890 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:52:01.890 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:52:01.890 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:52:01.890 INFO:tasks.workunit.client.0.trial144.stderr:131184: 2026-02-19T12:52:00.431+0000 7f78e32a0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:01.890 INFO:tasks.workunit.client.0.trial144.stderr:131184: 2026-02-19T12:52:00.431+0000 7f78e32a0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:01.890 INFO:tasks.workunit.client.0.trial144.stderr:131184: 2026-02-19T12:52:00.431+0000 7f78e32a0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:01.890 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T12:52:01.890 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:52:01.890 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:52:01.890 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:52:01.890 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:52:01.891 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:01.891 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:52:01.891 INFO:tasks.workunit.client.0.trial144.stderr:131184: 0 2026-02-19T12:52:01.891 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:01.891 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:52:01.891 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:01.891 INFO:tasks.workunit.client.0.trial144.stderr:131184: 2026-02-19T12:52:00.903+0000 7f78e32a0a40 -1 Falling back to public interface 2026-02-19T12:52:01.891 INFO:tasks.workunit.client.0.trial144.stderr:131184: 2026-02-19T12:52:01.627+0000 7f78e32a0a40 -1 osd.2 61 log_to_monitors true 2026-02-19T12:52:01.891 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:01.891 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:01.891 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:52:01.892 INFO:tasks.workunit.client.0.trial144.stderr:131184: 1 2026-02-19T12:52:01.892 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:01.892 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:52:01.892 INFO:tasks.workunit.client.0.trial144.stderr:131184: osd.2 up in weight 1 up_from 65 up_thru 46 down_at 62 last_clean_interval [59,61) [v2:127.0.0.1:6818/412031943,v1:127.0.0.1:6819/412031943] [v2:127.0.0.1:6820/412031943,v1:127.0.0.1:6821/412031943] exists,up 56b9b2cb-98f6-4b89-81c3-a3d557f1da96 2026-02-19T12:52:01.892 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:52:01.892 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:52:01.892 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:52:01.892 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:52:01.892 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:52:01.892 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:52:01.892 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:52:01.892 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:52:01.892 INFO:tasks.workunit.client.0.trial144.stderr:131184: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:52:01.893 INFO:tasks.workunit.client.0.trial144.stderr:131184: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1very_ops 2026-02-19T12:52:01.893 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:52:01.893 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:52:01.893 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:52:01.894 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:52:01.894 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:52:01.894 INFO:tasks.workunit.client.0.trial144.stderr:131182: 2026-02-19T12:52:00.423+0000 7f1f08dfaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:01.894 INFO:tasks.workunit.client.0.trial144.stderr:131182: 2026-02-19T12:52:00.423+0000 7f1f08dfaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:01.894 INFO:tasks.workunit.client.0.trial144.stderr:131182: 2026-02-19T12:52:00.423+0000 7f1f08dfaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:01.894 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:52:01.894 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:52:01.894 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:52:01.894 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:52:01.894 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:52:01.894 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:01.894 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:52:01.894 INFO:tasks.workunit.client.0.trial144.stderr:131182: 0 2026-02-19T12:52:01.895 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:01.895 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:52:01.895 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:01.895 INFO:tasks.workunit.client.0.trial144.stderr:131182: 2026-02-19T12:52:00.883+0000 7f1f08dfaa40 -1 Falling back to public interface 2026-02-19T12:52:01.895 INFO:tasks.workunit.client.0.trial144.stderr:131182: 2026-02-19T12:52:01.607+0000 7f1f08dfaa40 -1 osd.1 61 log_to_monitors true 2026-02-19T12:52:01.895 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:01.895 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:01.895 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:52:01.895 INFO:tasks.workunit.client.0.trial144.stderr:131182: 1 2026-02-19T12:52:01.895 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:01.895 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:52:01.895 INFO:tasks.workunit.client.0.trial144.stderr:131182: osd.1 up in weight 1 up_from 65 up_thru 65 down_at 62 last_clean_interval [59,61) [v2:127.0.0.1:6810/3203953585,v1:127.0.0.1:6811/3203953585] [v2:127.0.0.1:6812/3203953585,v1:127.0.0.1:6813/3203953585] exists,up 6a06dd6f-2a04-4584-96dd-5506f512ce78 2026-02-19T12:52:01.896 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:52:01.896 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:52:01.896 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:52:01.896 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:52:01.896 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:52:01.896 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:52:01.896 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:52:01.896 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:52:01.896 INFO:tasks.workunit.client.0.trial144.stderr:131182: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:52:02.448 INFO:tasks.workunit.client.0.trial144.stderr:131182: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:52:02.448 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:52:02.448 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:52:02.448 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:52:02.448 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:52:02.449 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:52:02.449 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:52:02.449 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:52:02.449 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:52:02.449 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:52:02.449 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:52:02.449 INFO:tasks.workunit.client.0.trial144.stderr:131182: 1 2026-02-19T12:52:02.449 INFO:tasks.workunit.client.0.trial144.stderr:131182: 2 2026-02-19T12:52:02.449 INFO:tasks.workunit.client.0.trial144.stderr:131182: 3' 2026-02-19T12:52:02.449 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:52:02.449 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:02.450 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:52:02.450 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836503 2026-02-19T12:52:02.450 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836503 2026-02-19T12:52:02.450 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503' 2026-02-19T12:52:02.450 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:02.450 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:52:02.450 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=279172874243 2026-02-19T12:52:02.450 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 279172874243 2026-02-19T12:52:02.450 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-279172874243' 2026-02-19T12:52:02.450 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:02.450 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:52:02.450 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=279172874243 2026-02-19T12:52:02.450 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 279172874243 2026-02-19T12:52:02.451 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-279172874243 2-279172874243' 2026-02-19T12:52:02.451 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:02.451 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:52:02.451 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=231928233992 2026-02-19T12:52:02.451 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 231928233992 2026-02-19T12:52:02.451 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-279172874243 2-279172874243 3-231928233992' 2026-02-19T12:52:02.451 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:02.451 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836503 2026-02-19T12:52:02.452 INFO:tasks.workunit.client.0.trial144.stderr:13118593: get_timeout_delays: echo true 2026-02-19T12:52:02.452 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:52:02.452 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:52:02.452 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:52:02.452 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:52:02.452 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:52:02.452 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:52:02.452 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:52:02.452 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:52:02.452 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:52:02.452 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:52:02.453 INFO:tasks.workunit.client.0.trial144.stderr:131184: 1 2026-02-19T12:52:02.453 INFO:tasks.workunit.client.0.trial144.stderr:131184: 2 2026-02-19T12:52:02.453 INFO:tasks.workunit.client.0.trial144.stderr:131184: 3' 2026-02-19T12:52:02.453 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:52:02.453 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:02.453 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:52:02.453 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836502 2026-02-19T12:52:02.453 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836502 2026-02-19T12:52:02.453 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502' 2026-02-19T12:52:02.453 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:02.453 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:52:02.453 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=279172874242 2026-02-19T12:52:02.453 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 279172874242 2026-02-19T12:52:02.454 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-279172874242' 2026-02-19T12:52:02.454 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:02.454 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:52:02.454 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=279172874242 2026-02-19T12:52:02.454 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 279172874242 2026-02-19T12:52:02.454 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-279172874242 2-279172874242' 2026-02-19T12:52:02.454 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:02.454 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:52:02.454 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=231928233993 2026-02-19T12:52:02.454 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 231928233993 2026-02-19T12:52:02.454 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-279172874242 2-279172874242 3-231928233993' 2026-02-19T12:52:02.454 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:02.455 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836502 2026-02-19T12:52:04.100 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:04.100 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:52:04.100 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836502 2026-02-19T12:52:04.100 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:04.100 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836502 2026-02-19T12:52:04.100 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836502' 2026-02-19T12:52:04.101 INFO:tasks.workunit.client.0.trial144.stderr:131184: waiting osd.0 seq 21474836502 2026-02-19T12:52:04.101 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:04.101 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836501 -lt 21474836502 2026-02-19T12:52:04.101 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:52:04.101 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:52:04.101 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:04.101 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836504 -lt 21474836502 2026-02-19T12:52:04.101 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:04.101 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-279172874242 2026-02-19T12:52:04.101 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:04.101 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:52:04.101 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-279172874242 2026-02-19T12:52:04.102 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:04.102 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=279172874242 2026-02-19T12:52:04.102 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 279172874242' 2026-02-19T12:52:04.102 INFO:tasks.workunit.client.0.trial144.stderr:131184: waiting osd.1 seq 279172874242 2026-02-19T12:52:04.102 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:52:04.102 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 279172874243 -lt 279172874242 2026-02-19T12:52:04.102 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:04.102 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-279172874242 2026-02-19T12:52:04.102 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:04.102 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:52:04.102 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-279172874242 2026-02-19T12:52:04.102 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:04.103 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=279172874242 2026-02-19T12:52:04.103 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 279172874242' 2026-02-19T12:52:04.103 INFO:tasks.workunit.client.0.trial144.stderr:131184: waiting osd.2 seq 279172874242 2026-02-19T12:52:04.103 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:52:04.103 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 279172874243 -lt 279172874242 2026-02-19T12:52:04.103 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:04.111 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-help2: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:04.111 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:52:04.111 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836503 2026-02-19T12:52:04.111 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:04.111 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836503 2026-02-19T12:52:04.111 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836503' 2026-02-19T12:52:04.111 INFO:tasks.workunit.client.0.trial144.stderr:131182: waiting osd.0 seq 21474836503 2026-02-19T12:52:04.112 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:04.112 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836501 -lt 21474836503 2026-02-19T12:52:04.112 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:52:04.112 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:52:04.112 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:04.112 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836504 -lt 21474836503 2026-02-19T12:52:04.112 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:04.112 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-279172874243 2026-02-19T12:52:04.112 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:04.112 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:52:04.112 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-279172874243 2026-02-19T12:52:04.112 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:04.112 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=279172874243 2026-02-19T12:52:04.113 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 279172874243' 2026-02-19T12:52:04.113 INFO:tasks.workunit.client.0.trial144.stderr:131182: waiting osd.1 seq 279172874243 2026-02-19T12:52:04.113 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:52:04.113 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 279172874243 -lt 279172874243 2026-02-19T12:52:04.113 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:04.113 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-279172874243 2026-02-19T12:52:04.113 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:04.113 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:52:04.113 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-279172874243 2026-02-19T12:52:04.113 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:04.113 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=279172874243 2026-02-19T12:52:04.113 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 279172874243' 2026-02-19T12:52:04.113 INFO:tasks.workunit.client.0.trial144.stderr:131182: waiting osd.2 seq 279172874243 2026-02-19T12:52:04.114 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:52:04.114 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 279172874243 -lt 279172874243 2026-02-19T12:52:04.114 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:04.803 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-ers.sh:2267: flush_pg_stats: echo 3-231928233993 2026-02-19T12:52:04.803 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-231928233993 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=231928233993 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 231928233993' 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: waiting osd.3 seq 231928233993 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 231928233994 -lt 231928233993 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:52:04.804 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:52:04.805 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:52:04.805 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:52:04.805 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:52:04.805 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:04.805 INFO:tasks.workunit.client.0.trial144.stderr:131184: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:04.805 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:52:04.805 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:52:04.805 INFO:tasks.workunit.client.0.trial144.stderr:131184: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:52:04.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:52:04.823 INFO:tasks.workunit.client.0.trial144.stderr:helpers.sh:2267: flush_pg_stats: echo 3-231928233992 2026-02-19T12:52:04.823 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:04.823 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:52:04.823 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-231928233992 2026-02-19T12:52:04.823 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:04.823 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=231928233992 2026-02-19T12:52:04.823 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 231928233992' 2026-02-19T12:52:04.823 INFO:tasks.workunit.client.0.trial144.stderr:131182: waiting osd.3 seq 231928233992 2026-02-19T12:52:04.823 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:52:04.823 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 231928233994 -lt 231928233992 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:52:04.824 INFO:tasks.workunit.client.0.trial144.stderr:131182: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:52:04.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:52:04.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:52:04.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 131180 2026-02-19T12:52:04.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:52:04.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:52:04.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:52:04.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:272: corrupt_and_repair_two: return_code=0 2026-02-19T12:52:04.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:273: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:52:04.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:275: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:52:04.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:276: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:52:04.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:323: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 3 1 2026-02-19T12:52:04.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:245: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-19T12:52:04.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:246: corrupt_and_repair_two: local poolname=ecpool 2026-02-19T12:52:04.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:247: corrupt_and_repair_two: local first=3 2026-02-19T12:52:04.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_two: local second=1 2026-02-19T12:52:04.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_two: pids= 2026-02-19T12:52:04.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:52:04.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:52:04.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:52:04.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 133452"' 2026-02-19T12:52:04.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 133452' 2026-02-19T12:52:04.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:52:04.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:52:04.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:52:04.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 133453"' 2026-02-19T12:52:04.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 133453' 2026-02-19T12:52:04.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_two: wait_background pids 2026-02-19T12:52:04.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 133452 133453' 2026-02-19T12:52:04.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:52:04.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:52:04.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 133452 2026-02-19T12:52:04.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:52:04.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/133455: /' 2026-02-19T12:52:04.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:52:04.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/133457: /' 2026-02-19T12:52:05.419 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:52:05.419 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:52:05.419 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:52:05.419 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:52:05.419 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:52:05.419 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:52:05.419 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:52:05.420 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:52:05.420 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:52:05.420 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:52:05.420 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:52:05.420 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:52:05.420 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:52:05.420 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:52:05.420 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:52:05.420 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:52:05.420 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:52:05.420 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:52:05.421 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:52:05.421 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:52:05.421 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:52:05.421 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:52:05.421 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-19T12:52:05.421 INFO:tasks.workunit.client.0.trial144.stderr:133455: remove 0#2:eb822e21:::SOMETHING:head# 2026-02-19T12:52:05.421 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:52:05.421 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:52:05.421 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:52:05.421 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:52:05.421 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:52:05.421 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:52:05.422 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:52:05.422 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:52:05.423 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:52:05.423 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:52:05.423 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:52:05.423 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:52:05.423 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:52:05.423 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:52:05.423 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:52:05.423 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:52:05.423 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:52:05.424 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:52:05.424 INFO:tasks.workunit.client.0.trial144.stderr:133455: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:52:05.424 INFO:tasks.workunit.client.0.trial144.stderr:133455: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:05.424 INFO:tasks.workunit.client.0.trial144.stderr:133455: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:05.424 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:05.424 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:05.424 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:52:05.424 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:52:05.424 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:52:05.424 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:52:05.424 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:52:05.424 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:52:05.424 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:52:05.425 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:52:05.425 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:52:05.425 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:52:05.425 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:52:05.425 INFO:tasks.workunit.client.0.trial144.stderr:133455: start osd.3 2026-02-19T12:52:05.425 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:52:05.439 INFO:tasks.workunit.client.0.trial144.stderr:1133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:52:05.439 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:52:05.440 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:52:05.440 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:52:05.440 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:52:05.440 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:52:05.440 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:52:05.440 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:52:05.440 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:52:05.440 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:52:05.440 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:52:05.440 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:52:05.440 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:52:05.441 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:52:05.441 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:52:05.441 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:52:05.441 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:52:05.441 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:52:05.441 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:52:05.441 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:52:05.441 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:52:05.441 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:52:05.441 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-19T12:52:05.441 INFO:tasks.workunit.client.0.trial144.stderr:133457: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-19T12:52:05.441 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:52:05.442 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:52:05.442 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:52:05.442 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:52:05.442 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:52:05.442 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:52:05.442 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:52:05.442 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:52:05.443 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:52:05.443 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:52:05.443 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:52:05.444 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:52:05.444 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:52:05.444 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:52:05.444 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:52:05.444 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:52:05.444 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:52:05.444 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:52:05.444 INFO:tasks.workunit.client.0.trial144.stderr:133457: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:52:05.444 INFO:tasks.workunit.client.0.trial144.stderr:133457: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:05.444 INFO:tasks.workunit.client.0.trial144.stderr:133457: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:05.444 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:05.444 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:05.444 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:52:05.445 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:52:05.445 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:52:05.445 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:52:05.445 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:52:05.445 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:52:05.445 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:52:05.445 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:52:05.445 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:52:05.445 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:52:05.445 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:52:05.445 INFO:tasks.workunit.client.0.trial144.stderr:133457: start osd.1 2026-02-19T12:52:05.445 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:52:06.904 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:52:06.904 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:52:06.904 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:52:06.904 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:52:06.905 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:52:06.905 INFO:tasks.workunit.client.0.trial144.stderr:133455: 2026-02-19T12:52:05.443+0000 7f73bacbea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:06.905 INFO:tasks.workunit.client.0.trial144.stderr:133455: 2026-02-19T12:52:05.443+0000 7f73bacbea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:06.905 INFO:tasks.workunit.client.0.trial144.stderr:133455: 2026-02-19T12:52:05.443+0000 7f73bacbea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:06.905 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:52:06.905 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:52:06.905 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:52:06.905 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:52:06.905 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:52:06.906 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:06.906 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:52:06.906 INFO:tasks.workunit.client.0.trial144.stderr:133455: 0 2026-02-19T12:52:06.906 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:06.906 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:52:06.906 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:06.907 INFO:tasks.workunit.client.0.trial144.stderr:133455: 2026-02-19T12:52:05.923+0000 7f73bacbea40 -1 Falling back to public interface 2026-02-19T12:52:06.907 INFO:tasks.workunit.client.0.trial144.stderr:133455: 2026-02-19T12:52:06.651+0000 7f73bacbea40 -1 osd.3 67 log_to_monitors true 2026-02-19T12:52:06.907 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:06.907 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:06.907 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:52:06.907 INFO:tasks.workunit.client.0.trial144.stderr:133455: 1 2026-02-19T12:52:06.907 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:06.907 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:52:06.907 INFO:tasks.workunit.client.0.trial144.stderr:133455: osd.3 up in weight 1 up_from 72 up_thru 65 down_at 68 last_clean_interval [54,67) [v2:127.0.0.1:6810/3729405476,v1:127.0.0.1:6811/3729405476] [v2:127.0.0.1:6812/3729405476,v1:127.0.0.1:6813/3729405476] exists,up f5a736f4-3482-41bf-9543-4ab259b1e387 2026-02-19T12:52:06.908 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:52:06.908 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:52:06.908 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:52:06.908 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:52:06.908 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:52:06.908 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:52:06.908 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:52:06.908 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:52:06.908 INFO:tasks.workunit.client.0.trial144.stderr:133455: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:52:07.466 INFO:tasks.workunit.client.0.trial144.stderr:133455: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:52:07.466 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:52:07.466 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:52:07.466 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:52:07.466 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:52:07.467 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:52:07.467 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:52:07.467 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:52:07.467 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:52:07.467 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:52:07.467 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:52:07.467 INFO:tasks.workunit.client.0.trial144.stderr:133455: 1 2026-02-19T12:52:07.467 INFO:tasks.workunit.client.0.trial144.stderr:133455: 2 2026-02-19T12:52:07.467 INFO:tasks.workunit.client.0.trial144.stderr:133455: 3' 2026-02-19T12:52:07.467 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:52:07.467 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:07.467 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:52:07.467 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836505 2026-02-19T12:52:07.467 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836505 2026-02-19T12:52:07.467 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505' 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=313532612610 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 313532612610 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-313532612610' 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=279172874245 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 279172874245 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-313532612610 2-279172874245' 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645314 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645314 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-313532612610 2-279172874245 3-309237645314' 2026-02-19T12:52:07.468 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:07.469 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836505 2026-02-19T12:52:08.083 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/33457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:52:08.084 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:52:08.084 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:52:08.084 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:52:08.084 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:52:08.084 INFO:tasks.workunit.client.0.trial144.stderr:133457: 2026-02-19T12:52:05.467+0000 7f84cfd7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:08.084 INFO:tasks.workunit.client.0.trial144.stderr:133457: 2026-02-19T12:52:05.467+0000 7f84cfd7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:08.084 INFO:tasks.workunit.client.0.trial144.stderr:133457: 2026-02-19T12:52:05.467+0000 7f84cfd7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:08.084 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:52:08.084 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:52:08.084 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:52:08.084 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:52:08.084 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:52:08.084 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:08.084 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:52:08.085 INFO:tasks.workunit.client.0.trial144.stderr:133457: 0 2026-02-19T12:52:08.085 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:08.085 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:52:08.085 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:08.085 INFO:tasks.workunit.client.0.trial144.stderr:133457: 2026-02-19T12:52:05.975+0000 7f84cfd7fa40 -1 Falling back to public interface 2026-02-19T12:52:08.085 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:08.085 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:08.085 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:52:08.085 INFO:tasks.workunit.client.0.trial144.stderr:133457: 1 2026-02-19T12:52:08.085 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:08.085 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:52:08.085 INFO:tasks.workunit.client.0.trial144.stderr:133457: 2026-02-19T12:52:06.783+0000 7f84cfd7fa40 -1 osd.1 67 log_to_monitors true 2026-02-19T12:52:08.085 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:08.085 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:08.085 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:08.086 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:52:08.086 INFO:tasks.workunit.client.0.trial144.stderr:133457: 2 2026-02-19T12:52:08.086 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:08.086 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:52:08.086 INFO:tasks.workunit.client.0.trial144.stderr:133457: osd.1 up in weight 1 up_from 73 up_thru 73 down_at 68 last_clean_interval [65,67) [v2:127.0.0.1:6826/4091423625,v1:127.0.0.1:6827/4091423625] [v2:127.0.0.1:6828/4091423625,v1:127.0.0.1:6829/4091423625] exists,up 6a06dd6f-2a04-4584-96dd-5506f512ce78 2026-02-19T12:52:08.086 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:52:08.086 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:52:08.086 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:52:08.086 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:52:08.575 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:52:08.576 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:52:08.576 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:52:08.576 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:52:08.576 INFO:tasks.workunit.client.0.trial144.stderr:133457: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:52:08.576 INFO:tasks.workunit.client.0.trial144.stderr:133457: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:52:08.576 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:52:08.576 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:52:08.576 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:52:08.576 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:52:08.576 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:52:08.576 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:52:08.577 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:52:08.577 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:52:08.577 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:52:08.577 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:52:08.577 INFO:tasks.workunit.client.0.trial144.stderr:133457: 1 2026-02-19T12:52:08.577 INFO:tasks.workunit.client.0.trial144.stderr:133457: 2 2026-02-19T12:52:08.577 INFO:tasks.workunit.client.0.trial144.stderr:133457: 3' 2026-02-19T12:52:08.577 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:52:08.577 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:08.577 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:52:08.577 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836507 2026-02-19T12:52:08.577 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836507 2026-02-19T12:52:08.578 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507' 2026-02-19T12:52:08.578 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:08.578 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:52:08.578 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=313532612611 2026-02-19T12:52:08.578 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 313532612611 2026-02-19T12:52:08.578 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-313532612611' 2026-02-19T12:52:08.578 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:08.578 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:52:08.578 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=279172874246 2026-02-19T12:52:08.578 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 279172874246 2026-02-19T12:52:08.578 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-313532612611 2-279172874246' 2026-02-19T12:52:08.578 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:10.136 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:52:10.136 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645315 2026-02-19T12:52:10.137 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645315 2026-02-19T12:52:10.137 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-313532612611 2-279172874246 3-309237645315' 2026-02-19T12:52:10.137 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:10.137 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836507 2026-02-19T12:52:10.137 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:10.137 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:52:10.137 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836507 2026-02-19T12:52:10.137 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:10.137 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836507 2026-02-19T12:52:10.137 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836507' 2026-02-19T12:52:10.137 INFO:tasks.workunit.client.0.trial144.stderr:133457: waiting osd.0 seq 21474836507 2026-02-19T12:52:10.138 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:10.138 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836504 -lt 21474836507 2026-02-19T12:52:10.138 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:52:10.138 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:52:10.138 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:10.138 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836507 -lt 21474836507 2026-02-19T12:52:10.138 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:10.138 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-313532612611 2026-02-19T12:52:10.138 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:10.138 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:52:10.138 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-313532612611 2026-02-19T12:52:10.138 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:10.139 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=313532612611 2026-02-19T12:52:10.139 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 313532612611' 2026-02-19T12:52:10.139 INFO:tasks.workunit.client.0.trial144.stderr:133457: waiting osd.1 seq 313532612611 2026-02-19T12:52:10.139 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:52:10.139 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 313532612611 -lt 313532612611 2026-02-19T12:52:10.139 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:10.139 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-279172874246 2026-02-19T12:52:10.139 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:10.139 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:52:10.139 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-279172874246 2026-02-19T12:52:10.139 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:10.144 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flushubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:10.144 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:52:10.144 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836505 2026-02-19T12:52:10.144 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:10.144 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836505 2026-02-19T12:52:10.144 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836505' 2026-02-19T12:52:10.144 INFO:tasks.workunit.client.0.trial144.stderr:133455: waiting osd.0 seq 21474836505 2026-02-19T12:52:10.145 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:10.145 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836504 -lt 21474836505 2026-02-19T12:52:10.145 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:52:10.145 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:52:10.145 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:10.145 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836504 -lt 21474836505 2026-02-19T12:52:10.145 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:52:10.145 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:52:10.145 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:10.145 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836507 -lt 21474836505 2026-02-19T12:52:10.145 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:10.145 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-313532612610 2026-02-19T12:52:10.146 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:10.146 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:52:10.146 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-313532612610 2026-02-19T12:52:10.146 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:10.146 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=313532612610 2026-02-19T12:52:10.146 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 313532612610' 2026-02-19T12:52:10.146 INFO:tasks.workunit.client.0.trial144.stderr:133455: waiting osd.1 seq 313532612610 2026-02-19T12:52:10.146 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:52:10.146 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 313532612611 -lt 313532612610 2026-02-19T12:52:10.146 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:10.146 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-279172874245 2026-02-19T12:52:10.146 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:10.146 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:52:10.147 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-279172874245 2026-02-19T12:52:10.147 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:10.147 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=279172874245 2026-02-19T12:52:10.147 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 279172874245' 2026-02-19T12:52:10.994 INFO:tasks.workunit.client.0.trial144.stderr:133455:_pg_stats: seq=279172874246 2026-02-19T12:52:10.995 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 279172874246' 2026-02-19T12:52:10.995 INFO:tasks.workunit.client.0.trial144.stderr:133457: waiting osd.2 seq 279172874246 2026-02-19T12:52:10.995 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:52:10.995 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 279172874246 -lt 279172874246 2026-02-19T12:52:10.995 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:10.995 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-309237645315 2026-02-19T12:52:10.995 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:10.995 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:52:10.995 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-309237645315 2026-02-19T12:52:10.995 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:10.996 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=309237645315 2026-02-19T12:52:10.996 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 309237645315' 2026-02-19T12:52:10.996 INFO:tasks.workunit.client.0.trial144.stderr:133457: waiting osd.3 seq 309237645315 2026-02-19T12:52:10.996 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:52:10.996 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645315 -lt 309237645315 2026-02-19T12:52:10.996 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:52:10.996 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:10.996 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:10.996 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:52:10.996 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:52:10.996 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:52:10.996 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:52:10.997 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:52:10.997 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:52:10.997 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:52:10.997 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:52:10.997 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:52:10.997 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:52:10.997 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:10.997 INFO:tasks.workunit.client.0.trial144.stderr:133457: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:10.997 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:52:10.997 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:52:10.997 INFO:tasks.workunit.client.0.trial144.stderr:133457: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:52:10.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:52:11.034 INFO:tasks.workunit.client.0.trial144.stderr: waiting osd.2 seq 279172874245 2026-02-19T12:52:11.034 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:52:11.034 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 279172874246 -lt 279172874245 2026-02-19T12:52:11.034 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:11.035 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-309237645314 2026-02-19T12:52:11.035 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:11.035 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:52:11.035 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-309237645314 2026-02-19T12:52:11.035 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:11.035 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=309237645314 2026-02-19T12:52:11.035 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 309237645314' 2026-02-19T12:52:11.035 INFO:tasks.workunit.client.0.trial144.stderr:133455: waiting osd.3 seq 309237645314 2026-02-19T12:52:11.035 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:52:11.035 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645315 -lt 309237645314 2026-02-19T12:52:11.035 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:52:11.036 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:11.036 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:11.036 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:52:11.036 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:52:11.036 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:52:11.036 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:52:11.036 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:52:11.036 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:52:11.036 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:52:11.036 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:52:11.036 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:52:11.036 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:52:11.036 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:11.037 INFO:tasks.workunit.client.0.trial144.stderr:133455: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:11.037 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:52:11.037 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:52:11.037 INFO:tasks.workunit.client.0.trial144.stderr:133455: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:52:11.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:52:11.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:52:11.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 133453 2026-02-19T12:52:11.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:52:11.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:52:11.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:52:11.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:257: corrupt_and_repair_two: return_code=0 2026-02-19T12:52:11.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:258: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:52:11.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-19T12:52:11.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:52:11.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:52:11.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:52:11.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:52:11.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: local pg=2.0 2026-02-19T12:52:11.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:264: corrupt_and_repair_two: repair 2.0 2026-02-19T12:52:11.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:52:11.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:52:11.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:52:11.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:52:11.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:52:11.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:52:11.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:51:53.611369+0000 2026-02-19T12:52:11.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:52:11.510 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:52:11.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:51:53.611369+0000 2026-02-19T12:52:11.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:52:11.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:51:53.611369+0000 2026-02-19T12:52:11.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:52:11.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:52:11.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:52:11.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:52:11.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:52:11.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:52:11.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:52:11.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:52:11.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:51:53.611369+0000 '>' 2026-02-19T12:51:53.611369+0000 2026-02-19T12:52:11.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:52:12.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:52:12.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:52:12.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:52:12.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:52:12.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:52:12.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:52:12.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:52:12.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:51:53.611369+0000 '>' 2026-02-19T12:51:53.611369+0000 2026-02-19T12:52:12.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:52:13.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:52:13.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:52:13.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:52:13.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:52:13.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:52:13.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:52:13.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:52:14.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:52:11.629380+0000 '>' 2026-02-19T12:51:53.611369+0000 2026-02-19T12:52:14.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:52:14.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:268: corrupt_and_repair_two: pids= 2026-02-19T12:52:14.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:269: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:52:14.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:52:14.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:52:14.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 136261"' 2026-02-19T12:52:14.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 136261' 2026-02-19T12:52:14.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:270: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:52:14.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:52:14.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:52:14.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 136262"' 2026-02-19T12:52:14.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 136262' 2026-02-19T12:52:14.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:271: corrupt_and_repair_two: wait_background pids 2026-02-19T12:52:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 136261 136262' 2026-02-19T12:52:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:52:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:52:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 136261 2026-02-19T12:52:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:52:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/136264: /' 2026-02-19T12:52:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:52:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/136266: /' 2026-02-19T12:52:14.494 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:52:14.495 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:52:14.495 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:52:14.495 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:52:14.495 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:52:14.495 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:52:14.495 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:52:14.495 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:52:14.495 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:52:14.495 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:52:14.496 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:52:14.496 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:52:14.496 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:52:14.496 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:52:14.496 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:52:14.496 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:52:14.496 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:52:14.496 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:52:14.496 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:52:14.496 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:52:14.496 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:52:14.496 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:52:14.496 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-19T12:52:14.497 INFO:tasks.workunit.client.0.trial144.stderr:136266: _ 2026-02-19T12:52:14.497 INFO:tasks.workunit.client.0.trial144.stderr:136266: hinfo_key 2026-02-19T12:52:14.497 INFO:tasks.workunit.client.0.trial144.stderr:136266: snapset 2026-02-19T12:52:14.497 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:52:14.497 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:52:14.497 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:52:14.497 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:52:14.497 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:52:14.497 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:52:14.497 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:52:14.497 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:52:14.498 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:52:14.498 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:52:14.498 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:52:14.498 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:52:14.498 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:52:14.499 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:52:14.499 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:52:14.499 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:52:14.499 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:52:14.499 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:52:14.499 INFO:tasks.workunit.client.0.trial144.stderr:136266: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:52:14.499 INFO:tasks.workunit.client.0.trial144.stderr:136266: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:14.499 INFO:tasks.workunit.client.0.trial144.stderr:136266: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:14.499 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:14.499 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:14.499 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:52:14.499 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:52:14.500 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:52:14.500 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:52:14.500 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:52:14.500 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:52:14.500 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:52:14.500 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:52:14.500 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:52:14.500 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:52:14.500 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:52:14.500 INFO:tasks.workunit.client.0.trial144.stderr:136266: start osd.1 2026-02-19T12:52:14.527 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_reco136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:52:14.527 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:52:14.527 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:52:14.527 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:52:14.527 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:52:14.527 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:52:14.527 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:52:14.528 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:52:14.528 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:52:14.528 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:52:14.528 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:52:14.528 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:52:14.528 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:52:14.528 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:52:14.528 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:52:14.528 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:52:14.528 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:52:14.528 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:52:14.528 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:52:14.529 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:52:14.529 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:52:14.529 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:52:14.529 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-19T12:52:14.529 INFO:tasks.workunit.client.0.trial144.stderr:136264: Error getting attr on : 2.0s0_head,0#-4:00000000:::deep_scrub_2.0s0:head#, (61) No data available 2026-02-19T12:52:14.529 INFO:tasks.workunit.client.0.trial144.stderr:136264: Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-19T12:52:14.529 INFO:tasks.workunit.client.0.trial144.stderr:136264: _ 2026-02-19T12:52:14.529 INFO:tasks.workunit.client.0.trial144.stderr:136264: hinfo_key 2026-02-19T12:52:14.529 INFO:tasks.workunit.client.0.trial144.stderr:136264: snapset 2026-02-19T12:52:14.529 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:52:14.529 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:52:14.529 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:52:14.530 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:52:14.530 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:52:14.530 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:52:14.530 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:52:14.530 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:52:14.530 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:52:14.530 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:52:14.531 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:52:14.531 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:52:14.531 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:52:14.531 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:52:14.531 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:52:14.531 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:52:14.531 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:52:14.531 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:52:14.531 INFO:tasks.workunit.client.0.trial144.stderr:136264: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:52:14.531 INFO:tasks.workunit.client.0.trial144.stderr:136264: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:14.531 INFO:tasks.workunit.client.0.trial144.stderr:136264: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:14.531 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:14.532 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:14.532 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:52:14.532 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:52:14.532 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:52:14.532 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:52:14.532 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:52:14.532 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:52:14.532 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:52:14.532 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:52:14.532 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:52:14.532 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:52:14.532 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:52:14.532 INFO:tasks.workunit.client.0.trial144.stderr:136264: start osd.3 2026-02-19T12:52:15.982 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.very_ops 2026-02-19T12:52:15.983 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:52:15.983 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:52:15.983 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:52:15.983 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:52:15.983 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:52:15.983 INFO:tasks.workunit.client.0.trial144.stderr:136266: 2026-02-19T12:52:14.514+0000 7f437ab33a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:15.983 INFO:tasks.workunit.client.0.trial144.stderr:136266: 2026-02-19T12:52:14.514+0000 7f437ab33a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:15.983 INFO:tasks.workunit.client.0.trial144.stderr:136266: 2026-02-19T12:52:14.518+0000 7f437ab33a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:15.983 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:52:15.984 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:52:15.984 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:52:15.984 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:52:15.984 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:52:15.984 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:15.984 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:52:15.984 INFO:tasks.workunit.client.0.trial144.stderr:136266: 0 2026-02-19T12:52:15.984 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:15.984 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:52:15.984 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:15.985 INFO:tasks.workunit.client.0.trial144.stderr:136266: 2026-02-19T12:52:14.986+0000 7f437ab33a40 -1 Falling back to public interface 2026-02-19T12:52:15.985 INFO:tasks.workunit.client.0.trial144.stderr:136266: 2026-02-19T12:52:15.662+0000 7f437ab33a40 -1 osd.1 74 log_to_monitors true 2026-02-19T12:52:15.985 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:15.985 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:15.985 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:52:15.985 INFO:tasks.workunit.client.0.trial144.stderr:136266: 1 2026-02-19T12:52:15.985 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:15.985 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:52:15.985 INFO:tasks.workunit.client.0.trial144.stderr:136266: osd.1 up in weight 1 up_from 78 up_thru 78 down_at 75 last_clean_interval [73,74) [v2:127.0.0.1:6810/3225773303,v1:127.0.0.1:6811/3225773303] [v2:127.0.0.1:6812/3225773303,v1:127.0.0.1:6813/3225773303] exists,up 6a06dd6f-2a04-4584-96dd-5506f512ce78 2026-02-19T12:52:15.985 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:52:15.985 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:52:15.985 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:52:15.986 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:52:15.986 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:52:15.986 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:52:15.986 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:52:15.986 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:52:15.986 INFO:tasks.workunit.client.0.trial144.stderr:136266: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:52:16.006 INFO:tasks.workunit.client.0.trial144.stderr:136266: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:52:16.006 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:52:16.006 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:52:16.006 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:52:16.006 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:52:16.006 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:52:16.006 INFO:tasks.workunit.client.0.trial144.stderr:136264: 2026-02-19T12:52:14.542+0000 7f979bf4da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:16.006 INFO:tasks.workunit.client.0.trial144.stderr:136264: 2026-02-19T12:52:14.542+0000 7f979bf4da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:16.006 INFO:tasks.workunit.client.0.trial144.stderr:136264: 2026-02-19T12:52:14.546+0000 7f979bf4da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:16.006 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:52:16.006 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:52:16.007 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:52:16.007 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:52:16.007 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:52:16.007 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:16.007 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:52:16.007 INFO:tasks.workunit.client.0.trial144.stderr:136264: 0 2026-02-19T12:52:16.007 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:16.007 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:52:16.007 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:16.007 INFO:tasks.workunit.client.0.trial144.stderr:136264: 2026-02-19T12:52:15.030+0000 7f979bf4da40 -1 Falling back to public interface 2026-02-19T12:52:16.007 INFO:tasks.workunit.client.0.trial144.stderr:136264: 2026-02-19T12:52:15.702+0000 7f979bf4da40 -1 osd.3 74 log_to_monitors true 2026-02-19T12:52:16.008 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:16.008 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:16.008 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:52:16.008 INFO:tasks.workunit.client.0.trial144.stderr:136264: 1 2026-02-19T12:52:16.008 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:16.008 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:52:16.008 INFO:tasks.workunit.client.0.trial144.stderr:136264: osd.3 up in weight 1 up_from 78 up_thru 78 down_at 75 last_clean_interval [72,74) [v2:127.0.0.1:6826/1578806992,v1:127.0.0.1:6827/1578806992] [v2:127.0.0.1:6828/1578806992,v1:127.0.0.1:6829/1578806992] exists,up f5a736f4-3482-41bf-9543-4ab259b1e387 2026-02-19T12:52:16.008 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:52:16.008 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:52:16.008 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:52:16.008 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:52:16.009 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:52:16.009 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:52:16.009 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:52:16.540 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delsh:1593: get_timeout_delays: echo true 2026-02-19T12:52:16.540 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:52:16.540 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:52:16.540 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:52:16.541 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:52:16.541 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:52:16.541 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:52:16.541 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:52:16.541 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:52:16.541 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:52:16.541 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:52:16.541 INFO:tasks.workunit.client.0.trial144.stderr:136266: 1 2026-02-19T12:52:16.541 INFO:tasks.workunit.client.0.trial144.stderr:136266: 2 2026-02-19T12:52:16.541 INFO:tasks.workunit.client.0.trial144.stderr:136266: 3' 2026-02-19T12:52:16.541 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:52:16.541 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:16.542 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:52:16.542 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836509 2026-02-19T12:52:16.542 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836509 2026-02-19T12:52:16.542 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509' 2026-02-19T12:52:16.542 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:16.542 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:52:16.542 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=335007449090 2026-02-19T12:52:16.542 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 335007449090 2026-02-19T12:52:16.542 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-335007449090' 2026-02-19T12:52:16.542 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:16.542 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:52:16.542 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=279172874248 2026-02-19T12:52:16.542 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 279172874248 2026-02-19T12:52:16.543 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-335007449090 2-279172874248' 2026-02-19T12:52:16.543 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:16.543 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:52:16.543 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=335007449090 2026-02-19T12:52:16.543 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 335007449090 2026-02-19T12:52:16.543 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-335007449090 2-279172874248 3-335007449090' 2026-02-19T12:52:16.543 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:16.543 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836509 2026-02-19T12:52:16.558 INFO:tasks.workunit.client.0.trial144.stderr:13626ays 90 .1 2026-02-19T12:52:16.558 INFO:tasks.workunit.client.0.trial144.stderr:136264: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:52:16.558 INFO:tasks.workunit.client.0.trial144.stderr:136264: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:52:16.558 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:52:16.558 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:52:16.558 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:52:16.559 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:52:16.559 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:52:16.559 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:52:16.559 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:52:16.559 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:52:16.559 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:52:16.559 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:52:16.559 INFO:tasks.workunit.client.0.trial144.stderr:136264: 1 2026-02-19T12:52:16.559 INFO:tasks.workunit.client.0.trial144.stderr:136264: 2 2026-02-19T12:52:16.559 INFO:tasks.workunit.client.0.trial144.stderr:136264: 3' 2026-02-19T12:52:16.559 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:52:16.559 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:16.560 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:52:16.560 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836510 2026-02-19T12:52:16.560 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836510 2026-02-19T12:52:16.560 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510' 2026-02-19T12:52:16.560 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:16.560 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:52:16.560 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=335007449091 2026-02-19T12:52:16.560 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 335007449091 2026-02-19T12:52:16.560 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-335007449091' 2026-02-19T12:52:16.560 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:16.560 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:52:16.560 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=279172874249 2026-02-19T12:52:16.560 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 279172874249 2026-02-19T12:52:16.560 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-335007449091 2-279172874249' 2026-02-19T12:52:16.561 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:16.561 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:52:16.561 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=335007449091 2026-02-19T12:52:16.561 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 335007449091 2026-02-19T12:52:16.561 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-335007449091 2-279172874249 3-335007449091' 2026-02-19T12:52:18.201 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest6: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:18.202 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:52:18.202 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836509 2026-02-19T12:52:18.202 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:18.202 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836509 2026-02-19T12:52:18.202 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836509' 2026-02-19T12:52:18.202 INFO:tasks.workunit.client.0.trial144.stderr:136266: waiting osd.0 seq 21474836509 2026-02-19T12:52:18.202 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:18.202 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836508 -lt 21474836509 2026-02-19T12:52:18.202 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:52:18.202 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:52:18.203 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:18.203 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836510 -lt 21474836509 2026-02-19T12:52:18.203 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:18.203 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-335007449090 2026-02-19T12:52:18.203 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:18.203 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:52:18.203 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-335007449090 2026-02-19T12:52:18.203 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:18.203 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=335007449090 2026-02-19T12:52:18.203 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 335007449090' 2026-02-19T12:52:18.203 INFO:tasks.workunit.client.0.trial144.stderr:136266: waiting osd.1 seq 335007449090 2026-02-19T12:52:18.203 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:52:18.204 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 335007449091 -lt 335007449090 2026-02-19T12:52:18.204 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:18.204 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-279172874248 2026-02-19T12:52:18.204 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:18.204 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:52:18.204 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-279172874248 2026-02-19T12:52:18.204 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:18.204 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=279172874248 2026-02-19T12:52:18.204 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 279172874248' 2026-02-19T12:52:18.204 INFO:tasks.workunit.client.0.trial144.stderr:136266: waiting osd.2 seq 279172874248 2026-02-19T12:52:18.204 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:52:18.204 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 279172874250 -lt 279172874248 2026-02-19T12:52:18.205 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:18.205 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:18.205 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836510 2026-02-19T12:52:18.205 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:18.205 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:52:18.205 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836510 2026-02-19T12:52:18.206 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:18.206 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836510 2026-02-19T12:52:18.206 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836510' 2026-02-19T12:52:18.206 INFO:tasks.workunit.client.0.trial144.stderr:136264: waiting osd.0 seq 21474836510 2026-02-19T12:52:18.206 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:18.206 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836508 -lt 21474836510 2026-02-19T12:52:18.206 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:52:18.206 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:52:18.206 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:18.206 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836510 -lt 21474836510 2026-02-19T12:52:18.206 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:18.206 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-335007449091 2026-02-19T12:52:18.207 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:18.207 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:52:18.207 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-335007449091 2026-02-19T12:52:18.207 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:18.207 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=335007449091 2026-02-19T12:52:18.207 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 335007449091' 2026-02-19T12:52:18.207 INFO:tasks.workunit.client.0.trial144.stderr:136264: waiting osd.1 seq 335007449091 2026-02-19T12:52:18.207 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:52:18.207 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 335007449091 -lt 335007449091 2026-02-19T12:52:18.207 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:18.207 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-279172874249 2026-02-19T12:52:18.207 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:18.207 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:52:18.208 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-279172874249 2026-02-19T12:52:18.208 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:18.208 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=279172874249 2026-02-19T12:52:18.208 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 279172874249' 2026-02-19T12:52:18.208 INFO:tasks.workunit.client.0.trial144.stderr:136264: waiting osd.2 seq 279172874249 2026-02-19T12:52:18.208 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:52:18.919 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 279172874250 -lt 279172874249 2026-02-19T12:52:18.919 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:18.920 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-335007449091 2026-02-19T12:52:18.920 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:18.920 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:52:18.920 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-335007449091 2026-02-19T12:52:18.920 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:18.920 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=335007449091 2026-02-19T12:52:18.920 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 335007449091' 2026-02-19T12:52:18.920 INFO:tasks.workunit.client.0.trial144.stderr:136264: waiting osd.3 seq 335007449091 2026-02-19T12:52:18.920 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:52:18.920 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 335007449091 -lt 335007449091 2026-02-19T12:52:18.921 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:52:18.921 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:18.921 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:18.921 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:52:18.921 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:52:18.921 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:52:18.921 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:52:18.921 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:52:18.921 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:52:18.922 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:52:18.922 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:52:18.922 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:52:18.922 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:52:18.922 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:18.922 INFO:tasks.workunit.client.0.trial144.stderr:136264: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:18.922 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:52:18.922 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:52:18.923 INFO:tasks.workunit.client.0.trial144.stderr:136264: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:52:18.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:52:18.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:52:18.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 136262 2026-02-19T12:52:18.925 INFO:tasks.workunit.client.0.trial144.stderr:helpers.sh:2267: flush_pg_stats: echo 3-335007449090 2026-02-19T12:52:18.925 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:18.926 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:52:18.926 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-335007449090 2026-02-19T12:52:18.926 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:18.926 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=335007449090 2026-02-19T12:52:18.926 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 335007449090' 2026-02-19T12:52:18.926 INFO:tasks.workunit.client.0.trial144.stderr:136266: waiting osd.3 seq 335007449090 2026-02-19T12:52:18.926 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:52:18.926 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 335007449091 -lt 335007449090 2026-02-19T12:52:18.926 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:52:18.926 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:18.927 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:18.927 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:52:18.927 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:52:18.927 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:52:18.927 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:52:18.927 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:52:18.927 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:52:18.927 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:52:18.927 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:52:18.927 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:52:18.927 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:52:18.928 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:18.928 INFO:tasks.workunit.client.0.trial144.stderr:136266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:18.928 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:52:18.928 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:52:18.928 INFO:tasks.workunit.client.0.trial144.stderr:136266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:52:18.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:52:18.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:52:18.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:52:18.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:52:18.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:272: corrupt_and_repair_two: return_code=0 2026-02-19T12:52:18.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:273: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:52:18.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:275: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:52:18.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:276: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:52:18.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T12:52:18.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:52:18.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:52:18.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:52:18.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:52:18.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:52:18.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:52:18.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:52:18.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:52:19.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:52:19.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:52:19.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:52:19.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:52:19.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:52:19.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:52:19.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:52:19.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:52:19.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:52:19.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:52:19.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:52:19.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:52:19.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:52:19.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:52:19.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:52:19.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:19.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:19.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:52:19.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:52:19.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:52:19.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T12:52:19.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T12:52:19.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T12:52:19.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T12:52:19.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:52:19.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:52:19.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:52:19.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:52:19.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:52:19.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:52:19.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:52:19.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:52:19.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:52:19.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:52:19.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:52:19.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:52:19.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:52:19.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:52:19.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:52:19.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:52:19.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:52:19.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:52:19.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:52:19.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:52:19.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:52:19.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:52:19.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:52:19.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:19.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:19.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:52:19.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:52:19.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:52:19.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T12:52:19.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:52:19.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:19.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:19.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T12:52:19.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:52:19.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T12:52:19.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:52:19.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T12:52:19.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_and_repair_jerasure_overwrites td/osd-scrub-repair 2026-02-19T12:52:19.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:878: TEST_corrupt_and_repair_jerasure_overwrites: '[' true = true ']' 2026-02-19T12:52:19.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:879: TEST_corrupt_and_repair_jerasure_overwrites: corrupt_and_repair_jerasure td/osd-scrub-repair true 2026-02-19T12:52:19.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:857: corrupt_and_repair_jerasure: local dir=td/osd-scrub-repair 2026-02-19T12:52:19.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:858: corrupt_and_repair_jerasure: local allow_overwrites=true 2026-02-19T12:52:19.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: corrupt_and_repair_jerasure: local poolname=ecpool 2026-02-19T12:52:19.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:861: corrupt_and_repair_jerasure: run_mon td/osd-scrub-repair a 2026-02-19T12:52:19.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T12:52:19.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T12:52:19.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T12:52:19.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T12:52:19.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T12:52:19.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T12:52:19.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T12:52:19.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:52:19.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:52:19.127 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:52:19.127 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:19.128 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:19.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:19.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T12:52:19.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T12:52:19.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T12:52:19.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:52:19.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:52:19.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T12:52:19.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T12:52:19.160 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:52:19.160 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:52:19.160 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:52:19.160 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:52:19.160 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:19.160 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:19.160 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:52:19.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:52:19.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T12:52:19.214 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T12:52:19.214 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:52:19.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:52:19.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T12:52:19.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T12:52:19.216 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:52:19.216 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:52:19.217 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:52:19.217 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:52:19.217 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:19.217 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:19.217 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:52:19.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:52:19.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T12:52:19.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:862: corrupt_and_repair_jerasure: run_mgr td/osd-scrub-repair x 2026-02-19T12:52:19.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T12:52:19.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T12:52:19.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T12:52:19.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T12:52:19.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T12:52:19.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T12:52:19.363 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T12:52:19.363 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:52:19.363 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:52:19.363 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:52:19.363 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:19.363 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:19.364 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:19.364 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:52:19.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:52:19.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:863: corrupt_and_repair_jerasure: seq 0 3 2026-02-19T12:52:19.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:863: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-19T12:52:19.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:864: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 0 2026-02-19T12:52:19.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:52:19.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:52:19.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T12:52:19.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:52:19.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:52:19.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:52:19.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:52:19.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:52:19.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:52:19.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:52:19.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:52:19.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:52:19.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:52:19.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:52:19.388 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:52:19.388 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:52:19.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:52:19.389 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:52:19.389 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:19.389 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:19.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:19.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:19.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:52:19.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:52:19.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:52:19.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:52:19.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:52:19.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:52:19.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:52:19.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:52:19.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:52:19.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:52:19.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:52:19.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:52:19.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:52:19.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4bb12cf9-1f81-47a9-9718-ad459964e280 2026-02-19T12:52:19.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 4bb12cf9-1f81-47a9-9718-ad459964e280' 2026-02-19T12:52:19.391 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 4bb12cf9-1f81-47a9-9718-ad459964e280 2026-02-19T12:52:19.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:52:19.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCDB5dpQqfqFxAANn3rlaiOJzSRBW8TuHimgg== 2026-02-19T12:52:19.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCDB5dpQqfqFxAANn3rlaiOJzSRBW8TuHimgg=="}' 2026-02-19T12:52:19.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4bb12cf9-1f81-47a9-9718-ad459964e280 -i td/osd-scrub-repair/0/new.json 2026-02-19T12:52:19.490 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:52:19.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T12:52:19.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCDB5dpQqfqFxAANn3rlaiOJzSRBW8TuHimgg== --osd-uuid 4bb12cf9-1f81-47a9-9718-ad459964e280 2026-02-19T12:52:19.516 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:19.514+0000 7f56e95f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:19.518 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:19.514+0000 7f56e95f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:19.519 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:19.518+0000 7f56e95f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:19.519 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:19.518+0000 7f56e95f1a40 -1 bdev(0x5653d8c42c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:52:19.519 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:19.518+0000 7f56e95f1a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T12:52:20.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T12:52:20.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:52:20.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T12:52:20.260 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:52:20.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:52:20.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T12:52:20.473 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:52:20.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:52:20.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:52:20.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:52:20.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:52:20.497 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:20.494+0000 7fea2786fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:20.498 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:20.494+0000 7fea2786fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:20.499 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:20.498+0000 7fea2786fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:20.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T12:52:20.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:52:20.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:52:20.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:52:20.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:52:20.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:20.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:52:20.636 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:52:20.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:20.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:52:20.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:20.884 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:20.882+0000 7fea2786fa40 -1 Falling back to public interface 2026-02-19T12:52:21.528 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:21.526+0000 7fea2786fa40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:52:21.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:21.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:21.798 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:52:21.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:52:21.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:21.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:52:21.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:22.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:22.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:22.958 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:52:22.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:52:22.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:22.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:52:23.121 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2135707330,v1:127.0.0.1:6803/2135707330] [v2:127.0.0.1:6804/2135707330,v1:127.0.0.1:6805/2135707330] exists,up 4bb12cf9-1f81-47a9-9718-ad459964e280 2026-02-19T12:52:23.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:52:23.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:52:23.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:52:23.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:863: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-19T12:52:23.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:864: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 1 2026-02-19T12:52:23.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:52:23.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:52:23.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T12:52:23.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:52:23.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:52:23.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:52:23.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:52:23.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:52:23.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:52:23.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:52:23.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:52:23.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:52:23.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:52:23.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:52:23.126 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:52:23.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:52:23.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:52:23.127 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:52:23.127 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:23.127 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:23.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:23.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:23.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:52:23.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:52:23.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:52:23.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:52:23.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:52:23.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:52:23.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:52:23.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:52:23.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:52:23.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:52:23.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:52:23.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:52:23.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:52:23.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d92ac76d-154c-47e9-851c-f60a9fbd2c0f 2026-02-19T12:52:23.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 d92ac76d-154c-47e9-851c-f60a9fbd2c0f' 2026-02-19T12:52:23.129 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 d92ac76d-154c-47e9-851c-f60a9fbd2c0f 2026-02-19T12:52:23.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:52:23.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCHB5dpEzc1CBAAUeUjd6w7uID29Lgps0jFCQ== 2026-02-19T12:52:23.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCHB5dpEzc1CBAAUeUjd6w7uID29Lgps0jFCQ=="}' 2026-02-19T12:52:23.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d92ac76d-154c-47e9-851c-f60a9fbd2c0f -i td/osd-scrub-repair/1/new.json 2026-02-19T12:52:23.298 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:52:23.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T12:52:23.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCHB5dpEzc1CBAAUeUjd6w7uID29Lgps0jFCQ== --osd-uuid d92ac76d-154c-47e9-851c-f60a9fbd2c0f 2026-02-19T12:52:23.324 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:23.322+0000 7fc4c44f0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:23.326 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:23.322+0000 7fc4c44f0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:23.327 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:23.326+0000 7fc4c44f0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:23.327 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:23.326+0000 7fc4c44f0a40 -1 bdev(0x55742881ac00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:52:23.327 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:23.326+0000 7fc4c44f0a40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T12:52:23.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T12:52:23.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:52:23.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T12:52:23.935 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:52:23.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:52:24.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T12:52:24.134 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:52:24.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:52:24.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:52:24.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:52:24.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:52:24.158 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:24.154+0000 7f576b06aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:24.160 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:24.158+0000 7f576b06aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:24.161 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:24.158+0000 7f576b06aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:24.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T12:52:24.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:52:24.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:52:24.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:52:24.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:52:24.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:24.296 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:52:24.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:52:24.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:24.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:52:24.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:24.524 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:24.522+0000 7f576b06aa40 -1 Falling back to public interface 2026-02-19T12:52:25.090 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:25.086+0000 7f576b06aa40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:52:25.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:25.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:25.455 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:52:25.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:52:25.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:25.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:52:25.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:26.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:26.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:26.616 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:52:26.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:52:26.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:26.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:52:26.770 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1455797052,v1:127.0.0.1:6811/1455797052] [v2:127.0.0.1:6812/1455797052,v1:127.0.0.1:6813/1455797052] exists,up d92ac76d-154c-47e9-851c-f60a9fbd2c0f 2026-02-19T12:52:26.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:52:26.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:52:26.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:52:26.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:863: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-19T12:52:26.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:864: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 2 2026-02-19T12:52:26.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:52:26.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:52:26.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T12:52:26.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:52:26.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:52:26.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:52:26.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:52:26.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:52:26.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:52:26.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:52:26.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:52:26.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:52:26.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:52:26.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:52:26.775 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:52:26.775 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:52:26.775 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:52:26.776 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:52:26.776 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:26.776 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:26.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:26.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:26.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:52:26.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:52:26.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:52:26.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:52:26.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:52:26.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:52:26.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:52:26.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:52:26.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:52:26.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:52:26.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:52:26.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:52:26.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:52:26.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4067a8a2-42ac-43fe-abfc-28e1d45c3c9e 2026-02-19T12:52:26.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 4067a8a2-42ac-43fe-abfc-28e1d45c3c9e' 2026-02-19T12:52:26.778 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 4067a8a2-42ac-43fe-abfc-28e1d45c3c9e 2026-02-19T12:52:26.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:52:26.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCKB5dpIJAaLxAA2Y6D28KoXraNhlU8VbA1/g== 2026-02-19T12:52:26.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCKB5dpIJAaLxAA2Y6D28KoXraNhlU8VbA1/g=="}' 2026-02-19T12:52:26.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4067a8a2-42ac-43fe-abfc-28e1d45c3c9e -i td/osd-scrub-repair/2/new.json 2026-02-19T12:52:26.950 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:52:26.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T12:52:26.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCKB5dpIJAaLxAA2Y6D28KoXraNhlU8VbA1/g== --osd-uuid 4067a8a2-42ac-43fe-abfc-28e1d45c3c9e 2026-02-19T12:52:26.977 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:26.974+0000 7f04016dba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:26.978 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:26.978+0000 7f04016dba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:26.979 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:26.978+0000 7f04016dba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:26.980 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:26.978+0000 7f04016dba40 -1 bdev(0x558fc66dec00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T12:52:26.980 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:26.978+0000 7f04016dba40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T12:52:27.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T12:52:27.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:52:27.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T12:52:27.628 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T12:52:27.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:52:27.829 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T12:52:27.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T12:52:27.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:52:27.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:52:27.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:52:27.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:52:27.853 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:27.850+0000 7f0652f97a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:27.855 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:27.854+0000 7f0652f97a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:27.856 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:27.854+0000 7f0652f97a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:27.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T12:52:27.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:52:27.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:52:27.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:52:27.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:52:27.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:27.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:52:27.994 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:52:27.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:27.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:52:28.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:28.228 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:28.226+0000 7f0652f97a40 -1 Falling back to public interface 2026-02-19T12:52:28.820 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:28.818+0000 7f0652f97a40 -1 osd.2 0 log_to_monitors true 2026-02-19T12:52:29.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:29.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:29.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:52:29.151 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:52:29.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:29.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:52:29.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:30.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:30.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:30.311 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:52:30.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:52:30.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:30.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:52:30.469 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2242156567,v1:127.0.0.1:6819/2242156567] [v2:127.0.0.1:6820/2242156567,v1:127.0.0.1:6821/2242156567] exists,up 4067a8a2-42ac-43fe-abfc-28e1d45c3c9e 2026-02-19T12:52:30.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:52:30.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:52:30.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:52:30.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:863: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-19T12:52:30.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:864: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 3 2026-02-19T12:52:30.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:52:30.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:52:30.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T12:52:30.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:52:30.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:52:30.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:52:30.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:52:30.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:52:30.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:52:30.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:52:30.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:52:30.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:52:30.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:52:30.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:52:30.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:52:30.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:52:30.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:52:30.474 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:52:30.474 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:30.474 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:30.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:30.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:30.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:52:30.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:52:30.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:52:30.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:52:30.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:52:30.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:52:30.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:52:30.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:52:30.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:52:30.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:52:30.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:52:30.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:52:30.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:52:30.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5b32660b-4e9c-40ea-9379-e74a0c3a6b89 2026-02-19T12:52:30.476 INFO:tasks.workunit.client.0.trial144.stdout:add osd3 5b32660b-4e9c-40ea-9379-e74a0c3a6b89 2026-02-19T12:52:30.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 5b32660b-4e9c-40ea-9379-e74a0c3a6b89' 2026-02-19T12:52:30.477 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:52:30.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCOB5dpuOgGHRAAUppM+Bho7bSV+yWnqo5Zfg== 2026-02-19T12:52:30.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCOB5dpuOgGHRAAUppM+Bho7bSV+yWnqo5Zfg=="}' 2026-02-19T12:52:30.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5b32660b-4e9c-40ea-9379-e74a0c3a6b89 -i td/osd-scrub-repair/3/new.json 2026-02-19T12:52:30.651 INFO:tasks.workunit.client.0.trial144.stdout:3 2026-02-19T12:52:30.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-19T12:52:30.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCOB5dpuOgGHRAAUppM+Bho7bSV+yWnqo5Zfg== --osd-uuid 5b32660b-4e9c-40ea-9379-e74a0c3a6b89 2026-02-19T12:52:30.688 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:30.686+0000 7ff459c3ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:30.689 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:30.686+0000 7ff459c3ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:30.690 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:30.686+0000 7ff459c3ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:30.691 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:30.690+0000 7ff459c3ea40 -1 bdev(0x55be49b82c00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-19T12:52:30.691 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:30.690+0000 7ff459c3ea40 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-19T12:52:31.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-19T12:52:31.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:52:31.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T12:52:31.359 INFO:tasks.workunit.client.0.trial144.stdout:adding osd3 key to auth repository 2026-02-19T12:52:31.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:52:31.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T12:52:31.562 INFO:tasks.workunit.client.0.trial144.stdout:start osd.3 2026-02-19T12:52:31.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:52:31.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:52:31.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:52:31.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:52:31.578 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:31.574+0000 7f75de5fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:31.579 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:31.578+0000 7f75de5fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:31.580 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:31.578+0000 7f75de5fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:31.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T12:52:31.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:52:31.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:52:31.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:52:31.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:52:31.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:31.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:52:31.720 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:52:31.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:31.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:52:31.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:31.964 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:31.962+0000 7f75de5fca40 -1 Falling back to public interface 2026-02-19T12:52:32.492 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:32.490+0000 7f75de5fca40 -1 osd.3 0 log_to_monitors true 2026-02-19T12:52:32.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:32.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:32.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:52:32.880 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:52:32.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:32.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:52:33.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:34.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:34.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:34.039 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:52:34.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:52:34.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:34.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:52:34.195 INFO:tasks.workunit.client.0.trial144.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/4099929572,v1:127.0.0.1:6827/4099929572] [v2:127.0.0.1:6828/4099929572,v1:127.0.0.1:6829/4099929572] exists,up 5b32660b-4e9c-40ea-9379-e74a0c3a6b89 2026-02-19T12:52:34.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:52:34.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:52:34.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:52:34.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:866: corrupt_and_repair_jerasure: create_rbd_pool 2026-02-19T12:52:34.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T12:52:34.346 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T12:52:34.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T12:52:34.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T12:52:34.575 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T12:52:34.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:52:35.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T12:52:35.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:867: corrupt_and_repair_jerasure: wait_for_clean 2026-02-19T12:52:35.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:52:35.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:52:35.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:52:35.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:52:35.864 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:52:35.864 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:52:35.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:52:35.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:52:35.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:52:35.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:52:35.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:52:35.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:52:35.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:52:35.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:52:35.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:52:36.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:52:36.140 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:52:36.140 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:52:36.140 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T12:52:36.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:52:36.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:36.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:52:36.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-19T12:52:36.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-19T12:52:36.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-19T12:52:36.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:36.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:52:36.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672964 2026-02-19T12:52:36.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672964 2026-02-19T12:52:36.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-42949672964' 2026-02-19T12:52:36.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:36.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:52:36.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542147 2026-02-19T12:52:36.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542147 2026-02-19T12:52:36.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-42949672964 2-60129542147' 2026-02-19T12:52:36.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:36.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:52:36.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378626 2026-02-19T12:52:36.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378626 2026-02-19T12:52:36.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-42949672964 2-60129542147 3-81604378626' 2026-02-19T12:52:36.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:36.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-19T12:52:36.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:36.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:52:36.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-19T12:52:36.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:36.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-19T12:52:36.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-19T12:52:36.434 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836484 2026-02-19T12:52:36.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:36.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-19T12:52:36.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:52:37.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:52:37.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:37.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-19T12:52:37.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:52:38.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:52:38.763 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:38.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836484 2026-02-19T12:52:38.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:38.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672964 2026-02-19T12:52:38.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:38.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:52:38.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672964 2026-02-19T12:52:38.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:38.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672964 2026-02-19T12:52:38.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-19T12:52:38.929 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672964 2026-02-19T12:52:38.930 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:52:39.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-19T12:52:39.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:39.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542147 2026-02-19T12:52:39.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:39.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:52:39.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542147 2026-02-19T12:52:39.100 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:39.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542147 2026-02-19T12:52:39.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-19T12:52:39.101 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542147 2026-02-19T12:52:39.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:52:39.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-19T12:52:39.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:39.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-81604378626 2026-02-19T12:52:39.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:39.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:52:39.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-81604378626 2026-02-19T12:52:39.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:39.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378626 2026-02-19T12:52:39.273 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.3 seq 81604378626 2026-02-19T12:52:39.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 81604378626' 2026-02-19T12:52:39.274 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:52:39.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378626 -lt 81604378626 2026-02-19T12:52:39.443 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:52:39.443 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:39.443 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:39.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:52:39.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:52:39.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:52:39.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:52:39.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:52:39.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:52:39.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:52:39.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:52:39.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:52:39.791 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:52:39.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:39.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:39.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:52:39.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:52:39.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:52:39.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:869: corrupt_and_repair_jerasure: create_ec_pool ecpool true k=2 m=2 2026-02-19T12:52:39.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: create_ec_pool: local pool_name=ecpool 2026-02-19T12:52:39.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: create_ec_pool: shift 2026-02-19T12:52:39.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: create_ec_pool: local allow_overwrites=true 2026-02-19T12:52:39.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2525: create_ec_pool: shift 2026-02-19T12:52:39.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=2 2026-02-19T12:52:40.189 INFO:tasks.workunit.client.0.trial144.stderr:load: isa 2026-02-19T12:52:40.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-19T12:52:40.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-19T12:52:40.494 INFO:tasks.workunit.client.0.trial144.stderr:pool 'ecpool' created 2026-02-19T12:52:40.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:52:41.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: create_ec_pool: '[' true = true ']' 2026-02-19T12:52:41.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2532: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-19T12:52:41.731 INFO:tasks.workunit.client.0.trial144.stderr:set pool 2 allow_ec_overwrites to true 2026-02-19T12:52:41.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2535: create_ec_pool: wait_for_clean 2026-02-19T12:52:41.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:52:41.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:52:41.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:52:41.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:52:41.746 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:52:41.746 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:52:41.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:52:41.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:52:41.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:52:41.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:52:41.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:52:41.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:52:41.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:52:41.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:52:41.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:52:42.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:52:42.017 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:52:42.017 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:52:42.018 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T12:52:42.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:52:42.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:42.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:52:42.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836486 2026-02-19T12:52:42.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836486 2026-02-19T12:52:42.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836486' 2026-02-19T12:52:42.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:42.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:52:42.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672966 2026-02-19T12:52:42.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672966 2026-02-19T12:52:42.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836486 1-42949672966' 2026-02-19T12:52:42.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:42.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:52:42.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542149 2026-02-19T12:52:42.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542149 2026-02-19T12:52:42.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836486 1-42949672966 2-60129542149' 2026-02-19T12:52:42.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:42.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:52:42.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378628 2026-02-19T12:52:42.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378628 2026-02-19T12:52:42.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836486 1-42949672966 2-60129542149 3-81604378628' 2026-02-19T12:52:42.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:42.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836486 2026-02-19T12:52:42.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:42.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:52:42.308 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836486 2026-02-19T12:52:42.308 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:42.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836486 2026-02-19T12:52:42.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-19T12:52:42.310 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836486 2026-02-19T12:52:42.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:42.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-19T12:52:42.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:52:43.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:52:43.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:43.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-19T12:52:43.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:52:44.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:52:44.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:44.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836486 2026-02-19T12:52:44.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:44.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672966 2026-02-19T12:52:44.793 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:44.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:52:44.794 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672966 2026-02-19T12:52:44.794 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:44.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672966 2026-02-19T12:52:44.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-19T12:52:44.796 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672966 2026-02-19T12:52:44.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:52:44.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672966 -lt 42949672966 2026-02-19T12:52:44.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:44.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542149 2026-02-19T12:52:44.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:44.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:52:44.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542149 2026-02-19T12:52:44.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:44.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542149 2026-02-19T12:52:44.958 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542149 2026-02-19T12:52:44.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542149' 2026-02-19T12:52:44.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:52:45.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542149 -lt 60129542149 2026-02-19T12:52:45.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:45.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-81604378628 2026-02-19T12:52:45.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:45.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:52:45.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-81604378628 2026-02-19T12:52:45.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:45.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378628 2026-02-19T12:52:45.132 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.3 seq 81604378628 2026-02-19T12:52:45.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 81604378628' 2026-02-19T12:52:45.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:52:45.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378629 -lt 81604378628 2026-02-19T12:52:45.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:52:45.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:45.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:45.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:52:45.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:52:45.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:52:45.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:52:45.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:52:45.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:52:45.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:52:45.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:52:45.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:52:45.637 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:52:45.637 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:45.637 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:45.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:52:45.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:52:45.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:52:45.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2536: create_ec_pool: return 0 2026-02-19T12:52:45.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:870: corrupt_and_repair_jerasure: corrupt_and_repair_erasure_coded td/osd-scrub-repair ecpool 2026-02-19T12:52:45.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:308: corrupt_and_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-19T12:52:45.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:309: corrupt_and_repair_erasure_coded: local poolname=ecpool 2026-02-19T12:52:45.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:311: corrupt_and_repair_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-19T12:52:45.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T12:52:45.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T12:52:45.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-19T12:52:45.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T12:52:45.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T12:52:45.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T12:52:46.006 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T12:52:46.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T12:52:46.208 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T12:52:46.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T12:52:46.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T12:52:46.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-19T12:52:46.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:313: corrupt_and_repair_erasure_coded: get_primary ecpool SOMETHING 2026-02-19T12:52:46.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=ecpool 2026-02-19T12:52:46.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:52:46.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:52:46.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:52:46.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:313: corrupt_and_repair_erasure_coded: local primary=3 2026-02-19T12:52:46.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: get_osds ecpool SOMETHING 2026-02-19T12:52:46.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: sed -e s/3// 2026-02-19T12:52:46.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T12:52:46.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T12:52:46.401 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:52:46.402 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T12:52:46.562 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T12:52:46.563 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:52:46.563 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:52:46.563 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T12:52:46.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 0 2026-02-19T12:52:46.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: osds=('1' '2' '0') 2026-02-19T12:52:46.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: local -a osds 2026-02-19T12:52:46.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:315: corrupt_and_repair_erasure_coded: local not_primary_first=1 2026-02-19T12:52:46.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:316: corrupt_and_repair_erasure_coded: local not_primary_second=2 2026-02-19T12:52:46.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:319: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 3 2026-02-19T12:52:46.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:286: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-19T12:52:46.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:287: corrupt_and_repair_one: local poolname=ecpool 2026-02-19T12:52:46.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:288: corrupt_and_repair_one: local osd=3 2026-02-19T12:52:46.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:293: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:52:46.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:52:46.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:52:46.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:52:46.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:52:46.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:52:46.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:52:46.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:52:46.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:52:46.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:52:46.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:52:46.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:52:46.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:52:46.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:52:46.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:52:46.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:52:46.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:52:46.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:52:46.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:52:46.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:52:46.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:52:46.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:52:46.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:52:46.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-19T12:52:47.055 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#2:eb822e21:::SOMETHING:head# 2026-02-19T12:52:47.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:52:47.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:52:47.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:52:47.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:52:47.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:52:47.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:52:47.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:52:47.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:52:47.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:52:47.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:52:47.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:52:47.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:52:47.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:52:47.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:52:47.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:52:47.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:52:47.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:52:47.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:52:47.158 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:52:47.158 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:47.158 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:47.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:47.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:47.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:52:47.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:52:47.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:52:47.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:52:47.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:52:47.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:52:47.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:52:47.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:52:47.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:52:47.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:52:47.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:52:47.161 INFO:tasks.workunit.client.0.trial144.stderr:start osd.3 2026-02-19T12:52:47.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:52:47.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:52:47.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:52:47.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:52:47.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:52:47.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:52:47.177 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:47.174+0000 7f7a1f609a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:47.179 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:47.178+0000 7f7a1f609a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:47.180 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:47.178+0000 7f7a1f609a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:47.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:52:47.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:52:47.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:52:47.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:52:47.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:52:47.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:47.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:52:47.317 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:52:47.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:47.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:52:47.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:47.588 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:47.586+0000 7f7a1f609a40 -1 Falling back to public interface 2026-02-19T12:52:48.208 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:48.206+0000 7f7a1f609a40 -1 osd.3 36 log_to_monitors true 2026-02-19T12:52:48.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:48.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:48.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:52:48.482 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:52:48.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:48.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:52:48.637 INFO:tasks.workunit.client.0.trial144.stderr:osd.3 up in weight 1 up_from 40 up_thru 40 down_at 37 last_clean_interval [19,36) [v2:127.0.0.1:6826/408593696,v1:127.0.0.1:6827/408593696] [v2:127.0.0.1:6828/408593696,v1:127.0.0.1:6829/408593696] exists,up 5b32660b-4e9c-40ea-9379-e74a0c3a6b89 2026-02-19T12:52:48.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:52:48.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:52:48.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:52:48.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:52:48.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:52:48.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:52:48.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:52:48.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:52:48.639 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:52:48.639 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:52:48.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:52:48.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:52:48.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:52:48.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:52:48.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:52:48.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:52:48.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:52:48.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:52:48.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:52:48.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:52:48.907 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:52:48.907 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:52:48.907 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T12:52:48.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:52:48.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:48.908 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:52:48.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836489 2026-02-19T12:52:48.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836489 2026-02-19T12:52:48.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489' 2026-02-19T12:52:48.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:48.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:52:49.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672968 2026-02-19T12:52:49.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672968 2026-02-19T12:52:49.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672968' 2026-02-19T12:52:49.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:49.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:52:49.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542151 2026-02-19T12:52:49.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542151 2026-02-19T12:52:49.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672968 2-60129542151' 2026-02-19T12:52:49.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:49.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:52:49.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=171798691842 2026-02-19T12:52:49.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 171798691842 2026-02-19T12:52:49.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672968 2-60129542151 3-171798691842' 2026-02-19T12:52:49.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:49.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836489 2026-02-19T12:52:49.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:49.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:52:49.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836489 2026-02-19T12:52:49.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:49.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836489 2026-02-19T12:52:49.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-19T12:52:49.201 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836489 2026-02-19T12:52:49.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:49.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-19T12:52:49.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:52:50.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:52:50.364 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:50.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-19T12:52:50.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:50.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672968 2026-02-19T12:52:50.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:50.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:52:50.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672968 2026-02-19T12:52:50.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:50.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672968 2026-02-19T12:52:50.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2026-02-19T12:52:50.526 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 42949672968 2026-02-19T12:52:50.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:52:50.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672968 -lt 42949672968 2026-02-19T12:52:50.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:50.690 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542151 2026-02-19T12:52:50.690 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:50.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:52:50.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542151 2026-02-19T12:52:50.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:50.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542151 2026-02-19T12:52:50.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542151' 2026-02-19T12:52:50.694 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542151 2026-02-19T12:52:50.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:52:50.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542151 -lt 60129542151 2026-02-19T12:52:50.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:50.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-171798691842 2026-02-19T12:52:50.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:50.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:52:50.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-171798691842 2026-02-19T12:52:50.868 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:50.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=171798691842 2026-02-19T12:52:50.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 171798691842' 2026-02-19T12:52:50.869 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 171798691842 2026-02-19T12:52:50.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:52:51.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 171798691842 -lt 171798691842 2026-02-19T12:52:51.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:52:51.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:51.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:51.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:52:51.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:52:51.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:52:51.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:52:51.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:52:51.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:52:51.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:52:51.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:52:51.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:52:51.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:52:51.387 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:51.387 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:51.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:52:51.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:52:51.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:52:51.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-19T12:52:51.573 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:52:51.573 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:52:51.573 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:52:51.573 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:52:51.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: local pg=2.0 2026-02-19T12:52:51.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: corrupt_and_repair_one: repair 2.0 2026-02-19T12:52:51.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:52:51.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:52:51.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:52:51.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:52:51.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:52:51.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:52:51.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:52:40.495215+0000 2026-02-19T12:52:51.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:52:52.032 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:52:52.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:52:40.495215+0000 2026-02-19T12:52:52.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:52:52.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:52:40.495215+0000 2026-02-19T12:52:52.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:52:52.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:52:52.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:52:52.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:52:52.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:52:52.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:52:52.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:52:52.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:52:52.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:52:40.495215+0000 '>' 2026-02-19T12:52:40.495215+0000 2026-02-19T12:52:52.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:52:53.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:52:53.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:52:53.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:52:53.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:52:53.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:52:53.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:52:53.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:52:53.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:52:40.495215+0000 '>' 2026-02-19T12:52:40.495215+0000 2026-02-19T12:52:53.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:52:54.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:52:54.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:52:54.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:52:54.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:52:54.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:52:54.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:52:54.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:52:54.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:52:52.211902+0000 '>' 2026-02-19T12:52:40.495215+0000 2026-02-19T12:52:54.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:52:54.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:302: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:52:54.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:52:54.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:52:54.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:52:54.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:52:54.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:52:54.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:52:54.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:52:54.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:52:54.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:52:54.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:52:54.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:52:54.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:52:54.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:52:54.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:52:54.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:52:54.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:52:54.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:52:54.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:52:54.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:52:54.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:52:54.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:52:54.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:52:54.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-19T12:52:54.737 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::deep_scrub_2.0s0:head#, (61) No data available 2026-02-19T12:52:54.737 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-19T12:52:54.737 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:52:54.737 INFO:tasks.workunit.client.0.trial144.stdout:hinfo_key 2026-02-19T12:52:54.737 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:52:54.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:52:54.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:52:54.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:52:54.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:52:54.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:52:54.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:52:54.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:52:54.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:52:54.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:52:54.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:52:54.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:52:54.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:52:54.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:52:54.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:52:54.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:52:54.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:52:54.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:52:54.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:52:54.835 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:52:54.835 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:54.835 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:54.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:54.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:54.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:52:54.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:52:54.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:52:54.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:52:54.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:52:54.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:52:54.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:52:54.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:52:54.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:52:54.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:52:54.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:52:54.837 INFO:tasks.workunit.client.0.trial144.stderr:start osd.3 2026-02-19T12:52:54.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:52:54.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:52:54.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:52:54.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:52:54.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:52:54.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:52:54.845 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:54.842+0000 7f1d9bf71a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:54.846 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:54.842+0000 7f1d9bf71a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:54.847 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:54.846+0000 7f1d9bf71a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:54.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:52:54.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:52:54.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:52:54.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:52:54.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:52:54.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:54.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:52:54.993 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:52:54.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:54.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:52:55.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:52:55.296 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:55.294+0000 7f1d9bf71a40 -1 Falling back to public interface 2026-02-19T12:52:55.916 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:55.914+0000 7f1d9bf71a40 -1 osd.3 42 log_to_monitors true 2026-02-19T12:52:56.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:52:56.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:52:56.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:52:56.152 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:52:56.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:52:56.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:52:56.308 INFO:tasks.workunit.client.0.trial144.stderr:osd.3 up in weight 1 up_from 46 up_thru 46 down_at 43 last_clean_interval [40,42) [v2:127.0.0.1:6826/3776828546,v1:127.0.0.1:6827/3776828546] [v2:127.0.0.1:6828/3776828546,v1:127.0.0.1:6829/3776828546] exists,up 5b32660b-4e9c-40ea-9379-e74a0c3a6b89 2026-02-19T12:52:56.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:52:56.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:52:56.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:52:56.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:52:56.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:52:56.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:52:56.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:52:56.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:52:56.309 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:52:56.309 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:52:56.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:52:56.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:52:56.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:52:56.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:52:56.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:52:56.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:52:56.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:52:56.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:52:56.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:52:56.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:52:56.578 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:52:56.578 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:52:56.578 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T12:52:56.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:52:56.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:56.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:52:56.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-19T12:52:56.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-19T12:52:56.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491' 2026-02-19T12:52:56.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:56.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:52:56.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672971 2026-02-19T12:52:56.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672971 2026-02-19T12:52:56.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672971' 2026-02-19T12:52:56.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:56.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:52:56.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542154 2026-02-19T12:52:56.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542154 2026-02-19T12:52:56.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672971 2-60129542154' 2026-02-19T12:52:56.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:52:56.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:52:56.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=197568495618 2026-02-19T12:52:56.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 197568495618 2026-02-19T12:52:56.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672971 2-60129542154 3-197568495618' 2026-02-19T12:52:56.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:56.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-19T12:52:56.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:56.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:52:56.876 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836491 2026-02-19T12:52:56.876 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:56.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836491 2026-02-19T12:52:56.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-19T12:52:56.878 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836491 2026-02-19T12:52:56.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:57.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T12:52:57.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:52:58.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:52:58.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:52:58.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836492 -lt 21474836491 2026-02-19T12:52:58.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:58.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672971 2026-02-19T12:52:58.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:58.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:52:58.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672971 2026-02-19T12:52:58.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:58.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672971 2026-02-19T12:52:58.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672971' 2026-02-19T12:52:58.214 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 42949672971 2026-02-19T12:52:58.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:52:58.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672971 -lt 42949672971 2026-02-19T12:52:58.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:58.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542154 2026-02-19T12:52:58.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:58.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:52:58.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542154 2026-02-19T12:52:58.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:58.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542154 2026-02-19T12:52:58.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542154' 2026-02-19T12:52:58.376 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542154 2026-02-19T12:52:58.376 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:52:58.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542154 -lt 60129542154 2026-02-19T12:52:58.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:52:58.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-197568495618 2026-02-19T12:52:58.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:52:58.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:52:58.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-197568495618 2026-02-19T12:52:58.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:52:58.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=197568495618 2026-02-19T12:52:58.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 197568495618' 2026-02-19T12:52:58.535 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 197568495618 2026-02-19T12:52:58.536 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:52:58.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 197568495618 -lt 197568495618 2026-02-19T12:52:58.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:52:58.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:58.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:58.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:52:58.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:52:58.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:52:58.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:52:58.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:52:58.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:52:58.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:52:58.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:52:59.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:52:59.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:52:59.051 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:52:59.051 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:52:59.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:52:59.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:52:59.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:52:59.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:52:59.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:304: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:52:59.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:321: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 1 2026-02-19T12:52:59.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:286: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-19T12:52:59.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:287: corrupt_and_repair_one: local poolname=ecpool 2026-02-19T12:52:59.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:288: corrupt_and_repair_one: local osd=1 2026-02-19T12:52:59.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:293: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:52:59.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:52:59.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:52:59.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:52:59.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:52:59.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:52:59.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:52:59.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:52:59.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:52:59.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:52:59.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:52:59.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:52:59.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:52:59.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:52:59.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:52:59.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:52:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:52:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:52:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:52:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:52:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:52:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:52:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:52:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-19T12:52:59.763 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-19T12:52:59.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:52:59.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:52:59.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:52:59.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:52:59.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:52:59.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:52:59.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:52:59.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:52:59.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:52:59.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:52:59.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:52:59.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:52:59.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:52:59.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:52:59.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:52:59.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:52:59.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:52:59.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:52:59.856 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:52:59.856 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:52:59.856 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:52:59.856 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:59.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:52:59.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:52:59.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:52:59.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:52:59.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:52:59.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:52:59.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:52:59.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:52:59.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:52:59.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:52:59.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:52:59.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:52:59.857 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:52:59.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:52:59.858 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:52:59.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:52:59.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:52:59.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:52:59.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:52:59.872 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:59.870+0000 7f9d1fc9ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:59.874 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:59.870+0000 7f9d1fc9ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:52:59.875 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:52:59.874+0000 7f9d1fc9ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:00.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:53:00.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:53:00.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:53:00.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:53:00.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:53:00.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:00.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:53:00.015 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:53:00.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:00.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:53:00.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:53:00.320 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:00.317+0000 7f9d1fc9ea40 -1 Falling back to public interface 2026-02-19T12:53:00.921 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:00.917+0000 7f9d1fc9ea40 -1 osd.1 47 log_to_monitors true 2026-02-19T12:53:01.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:53:01.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:01.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:53:01.175 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:53:01.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:01.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:53:01.331 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 51 up_thru 51 down_at 48 last_clean_interval [10,47) [v2:127.0.0.1:6810/1401659214,v1:127.0.0.1:6811/1401659214] [v2:127.0.0.1:6812/1401659214,v1:127.0.0.1:6813/1401659214] exists,up d92ac76d-154c-47e9-851c-f60a9fbd2c0f 2026-02-19T12:53:01.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:53:01.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:53:01.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:53:01.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:53:01.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:53:01.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:53:01.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:53:01.331 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:53:01.332 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:53:01.332 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:53:01.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:53:01.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:53:01.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:53:01.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:53:01.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:53:01.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:53:01.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:53:01.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:53:01.437 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:53:01.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:53:01.601 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:53:01.602 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:53:01.602 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T12:53:01.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:53:01.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:01.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:53:01.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836493 2026-02-19T12:53:01.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836493 2026-02-19T12:53:01.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493' 2026-02-19T12:53:01.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:01.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:53:01.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332098 2026-02-19T12:53:01.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332098 2026-02-19T12:53:01.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-219043332098' 2026-02-19T12:53:01.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:01.752 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:53:01.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542156 2026-02-19T12:53:01.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542156 2026-02-19T12:53:01.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-219043332098 2-60129542156' 2026-02-19T12:53:01.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:01.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:53:01.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=197568495620 2026-02-19T12:53:01.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 197568495620 2026-02-19T12:53:01.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-219043332098 2-60129542156 3-197568495620' 2026-02-19T12:53:01.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:01.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836493 2026-02-19T12:53:01.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:01.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:53:01.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836493 2026-02-19T12:53:01.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:01.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836493 2026-02-19T12:53:01.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-19T12:53:01.902 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836493 2026-02-19T12:53:01.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:02.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836493 -lt 21474836493 2026-02-19T12:53:02.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:02.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-219043332098 2026-02-19T12:53:02.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:02.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:53:02.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-219043332098 2026-02-19T12:53:02.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:02.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=219043332098 2026-02-19T12:53:02.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 219043332098' 2026-02-19T12:53:02.067 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 219043332098 2026-02-19T12:53:02.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:53:02.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 219043332098 -lt 219043332098 2026-02-19T12:53:02.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:02.237 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542156 2026-02-19T12:53:02.237 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:02.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:53:02.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542156 2026-02-19T12:53:02.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:02.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542156 2026-02-19T12:53:02.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542156' 2026-02-19T12:53:02.241 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542156 2026-02-19T12:53:02.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:53:02.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542156 -lt 60129542156 2026-02-19T12:53:02.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:02.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-197568495620 2026-02-19T12:53:02.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:02.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:53:02.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-197568495620 2026-02-19T12:53:02.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:02.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=197568495620 2026-02-19T12:53:02.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 197568495620' 2026-02-19T12:53:02.407 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 197568495620 2026-02-19T12:53:02.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:53:02.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 197568495619 -lt 197568495620 2026-02-19T12:53:02.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:53:03.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:53:03.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:53:03.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 197568495619 -lt 197568495620 2026-02-19T12:53:03.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:53:04.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:53:04.732 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:53:04.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 197568495620 -lt 197568495620 2026-02-19T12:53:04.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:53:04.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:04.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:05.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:53:05.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:05.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:05.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:05.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:05.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:05.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:05.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:05.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:53:05.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:05.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:05.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:05.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:53:05.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:53:05.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:53:05.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-19T12:53:05.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:53:05.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:53:05.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:53:05.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:53:05.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: local pg=2.0 2026-02-19T12:53:05.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: corrupt_and_repair_one: repair 2.0 2026-02-19T12:53:05.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:53:05.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:53:05.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:05.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:05.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:05.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:05.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:52:52.211902+0000 2026-02-19T12:53:05.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:53:05.896 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:53:05.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:52:52.211902+0000 2026-02-19T12:53:05.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:53:05.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:52:52.211902+0000 2026-02-19T12:53:05.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:53:05.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:53:05.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:05.907 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:05.907 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:05.907 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:05.907 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:05.907 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:06.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:52:52.211902+0000 '>' 2026-02-19T12:52:52.211902+0000 2026-02-19T12:53:06.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:53:07.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:53:07.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:07.070 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:07.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:07.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:07.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:07.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:07.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:52:52.211902+0000 '>' 2026-02-19T12:52:52.211902+0000 2026-02-19T12:53:07.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:53:08.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:53:08.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:08.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:08.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:08.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:08.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:08.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:08.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:52:52.211902+0000 '>' 2026-02-19T12:52:52.211902+0000 2026-02-19T12:53:08.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:53:09.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:53:09.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:09.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:09.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:09.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:09.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:09.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:09.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:52:52.211902+0000 '>' 2026-02-19T12:52:52.211902+0000 2026-02-19T12:53:09.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:53:10.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:53:10.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:10.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:10.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:10.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:10.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:10.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:10.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:52:52.211902+0000 '>' 2026-02-19T12:52:52.211902+0000 2026-02-19T12:53:10.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:53:11.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:53:11.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:11.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:11.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:11.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:11.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:11.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:11.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:53:06.875410+0000 '>' 2026-02-19T12:52:52.211902+0000 2026-02-19T12:53:11.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:53:11.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:302: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:53:11.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:53:11.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:53:11.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:53:11.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:53:11.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:53:11.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:53:11.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:53:11.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:53:11.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:53:11.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:53:11.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:53:11.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:53:11.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:53:11.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:53:11.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:53:12.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:53:12.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:53:12.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:53:12.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:53:12.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:53:12.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:53:12.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:53:12.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-19T12:53:12.240 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::deep_scrub_2.0s0:head#, (61) No data available 2026-02-19T12:53:12.241 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-19T12:53:12.241 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:53:12.241 INFO:tasks.workunit.client.0.trial144.stdout:hinfo_key 2026-02-19T12:53:12.241 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:53:12.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:53:12.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:53:12.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:53:12.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:53:12.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:53:12.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:53:12.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:53:12.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:53:12.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:53:12.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:53:12.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:53:12.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:53:12.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:53:12.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:53:12.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:53:12.337 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:53:12.337 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:53:12.337 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:53:12.337 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:53:12.337 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:12.337 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:12.337 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:12.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:12.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:53:12.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:53:12.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:53:12.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:53:12.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:53:12.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:53:12.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:53:12.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:53:12.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:53:12.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:53:12.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:53:12.340 INFO:tasks.workunit.client.0.trial144.stderr:start osd.3 2026-02-19T12:53:12.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:53:12.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:53:12.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:53:12.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:53:12.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:53:12.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:53:12.357 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:12.353+0000 7fdaa088ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:12.358 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:12.357+0000 7fdaa088ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:12.359 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:12.357+0000 7fdaa088ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:12.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:53:12.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:53:12.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:53:12.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:53:12.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:53:12.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:12.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:53:12.501 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:53:12.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:12.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:53:12.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:53:12.700 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:12.697+0000 7fdaa088ba40 -1 Falling back to public interface 2026-02-19T12:53:13.253 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:13.249+0000 7fdaa088ba40 -1 osd.3 53 log_to_monitors true 2026-02-19T12:53:13.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:53:13.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:13.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:53:13.663 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:53:13.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:13.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:53:13.816 INFO:tasks.workunit.client.0.trial144.stderr:osd.3 up in weight 1 up_from 57 up_thru 57 down_at 54 last_clean_interval [46,53) [v2:127.0.0.1:6826/3459453190,v1:127.0.0.1:6827/3459453190] [v2:127.0.0.1:6828/3459453190,v1:127.0.0.1:6829/3459453190] exists,up 5b32660b-4e9c-40ea-9379-e74a0c3a6b89 2026-02-19T12:53:13.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:53:13.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:53:13.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:53:13.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:53:13.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:53:13.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:53:13.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:53:13.817 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:53:13.817 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:53:13.818 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:53:13.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:53:13.819 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:53:13.820 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:53:13.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:53:13.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:53:13.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:53:13.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:53:13.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:53:13.931 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:53:14.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:53:14.088 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:53:14.088 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:53:14.088 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T12:53:14.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:53:14.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:14.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:53:14.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836497 2026-02-19T12:53:14.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836497 2026-02-19T12:53:14.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497' 2026-02-19T12:53:14.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:14.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:53:14.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332101 2026-02-19T12:53:14.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332101 2026-02-19T12:53:14.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-219043332101' 2026-02-19T12:53:14.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:14.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:53:14.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542159 2026-02-19T12:53:14.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542159 2026-02-19T12:53:14.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-219043332101 2-60129542159' 2026-02-19T12:53:14.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:14.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:53:14.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=244813135874 2026-02-19T12:53:14.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 244813135874 2026-02-19T12:53:14.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-219043332101 2-60129542159 3-244813135874' 2026-02-19T12:53:14.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:14.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836497 2026-02-19T12:53:14.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:14.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:53:14.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836497 2026-02-19T12:53:14.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:14.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836497 2026-02-19T12:53:14.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2026-02-19T12:53:14.383 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836497 2026-02-19T12:53:14.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:14.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836496 -lt 21474836497 2026-02-19T12:53:14.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:53:15.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:53:15.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:15.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836496 -lt 21474836497 2026-02-19T12:53:15.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:53:16.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:53:16.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:16.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836497 -lt 21474836497 2026-02-19T12:53:16.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:16.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-219043332101 2026-02-19T12:53:16.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:16.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:53:16.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-219043332101 2026-02-19T12:53:16.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:16.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=219043332101 2026-02-19T12:53:16.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 219043332101' 2026-02-19T12:53:16.872 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 219043332101 2026-02-19T12:53:16.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:53:17.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 219043332101 -lt 219043332101 2026-02-19T12:53:17.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:17.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542159 2026-02-19T12:53:17.029 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:17.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:53:17.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542159 2026-02-19T12:53:17.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:17.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542159 2026-02-19T12:53:17.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542159' 2026-02-19T12:53:17.032 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542159 2026-02-19T12:53:17.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:53:17.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542160 -lt 60129542159 2026-02-19T12:53:17.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:17.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-244813135874 2026-02-19T12:53:17.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:17.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:53:17.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-244813135874 2026-02-19T12:53:17.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:17.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=244813135874 2026-02-19T12:53:17.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 244813135874' 2026-02-19T12:53:17.205 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 244813135874 2026-02-19T12:53:17.205 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:53:17.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 244813135874 -lt 244813135874 2026-02-19T12:53:17.376 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:53:17.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:17.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:17.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:53:17.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:17.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:17.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:17.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:17.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:17.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:17.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:17.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:53:17.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:17.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:17.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:17.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:53:17.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:53:17.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:53:17.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:53:17.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:304: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:53:17.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:322: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 1 2 2026-02-19T12:53:17.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:245: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-19T12:53:17.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:246: corrupt_and_repair_two: local poolname=ecpool 2026-02-19T12:53:17.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:247: corrupt_and_repair_two: local first=1 2026-02-19T12:53:17.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_two: local second=2 2026-02-19T12:53:17.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_two: pids= 2026-02-19T12:53:17.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:53:17.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:53:17.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:53:17.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 150973"' 2026-02-19T12:53:17.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 150973' 2026-02-19T12:53:17.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T12:53:17.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:53:17.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:53:17.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 150975"' 2026-02-19T12:53:17.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 150975' 2026-02-19T12:53:17.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_two: wait_background pids 2026-02-19T12:53:17.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 150973 150975' 2026-02-19T12:53:17.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:53:17.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:53:17.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 150973 2026-02-19T12:53:17.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:53:17.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/150976: /' 2026-02-19T12:53:17.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/150978: /' 2026-02-19T12:53:17.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T12:53:18.555 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:53:18.555 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:53:18.555 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:53:18.555 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:53:18.555 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:53:18.556 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:53:18.556 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:53:18.556 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:53:18.556 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:53:18.556 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:53:18.556 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:53:18.556 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:53:18.556 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:53:18.556 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:53:18.556 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:53:18.556 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:53:18.556 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:53:18.557 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:53:18.557 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:53:18.557 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:53:18.557 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:53:18.557 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:53:18.557 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-19T12:53:18.557 INFO:tasks.workunit.client.0.trial144.stderr:150976: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-19T12:53:18.557 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:53:18.557 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:53:18.557 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:53:18.557 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:53:18.558 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:53:18.558 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:53:18.558 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:53:18.558 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:53:18.559 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:53:18.559 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:53:18.559 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:53:18.559 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:53:18.559 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:53:18.559 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:53:18.559 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:53:18.559 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:53:18.559 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:53:18.560 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:53:18.560 INFO:tasks.workunit.client.0.trial144.stderr:150976: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:53:18.560 INFO:tasks.workunit.client.0.trial144.stderr:150976: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:18.560 INFO:tasks.workunit.client.0.trial144.stderr:150976: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:18.560 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:18.560 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:18.560 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:53:18.560 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:53:18.560 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:53:18.560 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:53:18.560 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:53:18.561 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:53:18.561 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:53:18.561 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:53:18.561 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:53:18.561 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:53:18.561 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:53:18.561 INFO:tasks.workunit.client.0.trial144.stderr:150976: start osd.1 2026-02-19T12:53:18.561 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:53:18.594 INFO:tasks.workunit.client.0.trial144.stderr:1150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:53:18.594 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:53:18.594 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T12:53:18.594 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:53:18.594 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T12:53:18.594 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:53:18.594 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING remove 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: remove 2#2:eb822e21:::SOMETHING:head# 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-19T12:53:18.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:53:18.596 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:53:18.596 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T12:53:18.596 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:53:18.596 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:53:18.596 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:53:18.596 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:53:18.597 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:53:18.598 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:53:18.598 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:53:18.598 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:53:18.598 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:53:18.598 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:53:18.598 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:53:18.598 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:53:18.598 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:53:18.598 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:53:18.598 INFO:tasks.workunit.client.0.trial144.stderr:150978: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:53:18.598 INFO:tasks.workunit.client.0.trial144.stderr:150978: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:18.598 INFO:tasks.workunit.client.0.trial144.stderr:150978: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:18.598 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:18.599 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:18.599 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:53:18.599 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:53:18.599 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:53:18.599 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:53:18.599 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:53:18.599 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:53:18.599 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:53:18.599 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:53:18.599 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:53:18.600 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:53:18.600 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T12:53:18.600 INFO:tasks.workunit.client.0.trial144.stderr:150978: start osd.2 2026-02-19T12:53:18.600 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:53:20.041 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:53:20.041 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:53:20.041 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:53:20.041 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:53:20.041 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:53:20.041 INFO:tasks.workunit.client.0.trial144.stderr:150976: 2026-02-19T12:53:18.573+0000 7fa913e0ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:20.041 INFO:tasks.workunit.client.0.trial144.stderr:150976: 2026-02-19T12:53:18.577+0000 7fa913e0ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:20.041 INFO:tasks.workunit.client.0.trial144.stderr:150976: 2026-02-19T12:53:18.577+0000 7fa913e0ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:20.041 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:53:20.042 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:53:20.042 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:53:20.042 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:53:20.042 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:53:20.042 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:20.042 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:53:20.042 INFO:tasks.workunit.client.0.trial144.stderr:150976: 0 2026-02-19T12:53:20.042 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:20.042 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:53:20.042 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:53:20.042 INFO:tasks.workunit.client.0.trial144.stderr:150976: 2026-02-19T12:53:19.025+0000 7fa913e0ba40 -1 Falling back to public interface 2026-02-19T12:53:20.042 INFO:tasks.workunit.client.0.trial144.stderr:150976: 2026-02-19T12:53:19.665+0000 7fa913e0ba40 -1 osd.1 59 log_to_monitors true 2026-02-19T12:53:20.043 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:53:20.043 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:20.043 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:53:20.043 INFO:tasks.workunit.client.0.trial144.stderr:150976: 1 2026-02-19T12:53:20.043 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:20.043 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:53:20.043 INFO:tasks.workunit.client.0.trial144.stderr:150976: osd.1 up in weight 1 up_from 63 up_thru 63 down_at 60 last_clean_interval [51,59) [v2:127.0.0.1:6810/832880183,v1:127.0.0.1:6811/832880183] [v2:127.0.0.1:6812/832880183,v1:127.0.0.1:6813/832880183] exists,up d92ac76d-154c-47e9-851c-f60a9fbd2c0f 2026-02-19T12:53:20.043 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:53:20.043 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:53:20.043 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:53:20.043 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:53:20.043 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:53:20.044 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:53:20.044 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:53:20.044 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:53:20.044 INFO:tasks.workunit.client.0.trial144.stderr:150976: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:53:20.071 INFO:tasks.workunit.client.0.trial144.stderr:150976: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_t50978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-19T12:53:20.071 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T12:53:20.071 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:53:20.071 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:53:20.071 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:53:20.071 INFO:tasks.workunit.client.0.trial144.stderr:150978: 2026-02-19T12:53:18.609+0000 7fa1e82aba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:20.072 INFO:tasks.workunit.client.0.trial144.stderr:150978: 2026-02-19T12:53:18.609+0000 7fa1e82aba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:20.072 INFO:tasks.workunit.client.0.trial144.stderr:150978: 2026-02-19T12:53:18.609+0000 7fa1e82aba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:20.072 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T12:53:20.072 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:53:20.072 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:53:20.072 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:53:20.072 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:53:20.072 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:20.072 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:53:20.072 INFO:tasks.workunit.client.0.trial144.stderr:150978: 0 2026-02-19T12:53:20.072 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:20.072 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:53:20.073 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:53:20.073 INFO:tasks.workunit.client.0.trial144.stderr:150978: 2026-02-19T12:53:19.037+0000 7fa1e82aba40 -1 Falling back to public interface 2026-02-19T12:53:20.073 INFO:tasks.workunit.client.0.trial144.stderr:150978: 2026-02-19T12:53:19.657+0000 7fa1e82aba40 -1 osd.2 60 log_to_monitors true 2026-02-19T12:53:20.073 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:53:20.073 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:20.073 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:53:20.073 INFO:tasks.workunit.client.0.trial144.stderr:150978: 1 2026-02-19T12:53:20.073 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:20.073 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:53:20.073 INFO:tasks.workunit.client.0.trial144.stderr:150978: osd.2 up in weight 1 up_from 63 up_thru 48 down_at 60 last_clean_interval [14,60) [v2:127.0.0.1:6818/4157769422,v1:127.0.0.1:6819/4157769422] [v2:127.0.0.1:6820/4157769422,v1:127.0.0.1:6821/4157769422] exists,up 4067a8a2-42ac-43fe-abfc-28e1d45c3c9e 2026-02-19T12:53:20.073 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:53:20.073 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:53:20.074 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:53:20.074 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:53:20.074 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:53:20.074 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:53:20.074 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:53:20.074 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:53:20.074 INFO:tasks.workunit.client.0.trial144.stderr:150978: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:53:20.595 INFO:tasks.workunit.client.0.trial144.stderr:150978: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: gimeout_delays: echo true 2026-02-19T12:53:20.595 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:53:20.595 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:53:20.595 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:53:20.595 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:53:20.596 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:53:20.596 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:53:20.596 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:53:20.596 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:53:20.596 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:53:20.596 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:53:20.596 INFO:tasks.workunit.client.0.trial144.stderr:150976: 1 2026-02-19T12:53:20.596 INFO:tasks.workunit.client.0.trial144.stderr:150976: 2 2026-02-19T12:53:20.596 INFO:tasks.workunit.client.0.trial144.stderr:150976: 3' 2026-02-19T12:53:20.596 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:53:20.596 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:20.597 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:53:20.597 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836499 2026-02-19T12:53:20.597 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836499 2026-02-19T12:53:20.597 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499' 2026-02-19T12:53:20.597 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:20.597 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:53:20.597 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=270582939650 2026-02-19T12:53:20.597 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 270582939650 2026-02-19T12:53:20.597 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-270582939650' 2026-02-19T12:53:20.597 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:20.597 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:53:20.597 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=270582939650 2026-02-19T12:53:20.597 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 270582939650 2026-02-19T12:53:20.598 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-270582939650 2-270582939650' 2026-02-19T12:53:20.598 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:20.598 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:53:20.598 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=244813135876 2026-02-19T12:53:20.598 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 244813135876 2026-02-19T12:53:20.598 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-270582939650 2-270582939650 3-244813135876' 2026-02-19T12:53:20.598 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:20.598 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836499 2026-02-19T12:53:20.617 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubunet_timeout_delays: echo true 2026-02-19T12:53:20.617 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:53:20.617 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:53:20.617 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:53:20.617 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:53:20.617 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:53:20.617 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:53:20.617 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:53:20.618 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:53:20.618 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:53:20.618 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:53:20.618 INFO:tasks.workunit.client.0.trial144.stderr:150978: 1 2026-02-19T12:53:20.618 INFO:tasks.workunit.client.0.trial144.stderr:150978: 2 2026-02-19T12:53:20.618 INFO:tasks.workunit.client.0.trial144.stderr:150978: 3' 2026-02-19T12:53:20.618 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:53:20.618 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:20.618 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:53:20.618 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836500 2026-02-19T12:53:20.618 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836500 2026-02-19T12:53:20.618 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836500' 2026-02-19T12:53:20.619 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:20.619 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:53:20.619 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=270582939651 2026-02-19T12:53:20.619 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 270582939651 2026-02-19T12:53:20.619 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836500 1-270582939651' 2026-02-19T12:53:20.619 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:20.619 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:53:20.619 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=270582939651 2026-02-19T12:53:20.619 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 270582939651 2026-02-19T12:53:20.619 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836500 1-270582939651 2-270582939651' 2026-02-19T12:53:20.619 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:20.619 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:53:20.619 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=244813135877 2026-02-19T12:53:20.620 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 244813135877 2026-02-19T12:53:20.620 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836500 1-270582939651 2-270582939651 3-244813135877' 2026-02-19T12:53:20.620 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:20.620 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836500 2026-02-19T12:53:22.249 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/tu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:22.249 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:53:22.249 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836499 2026-02-19T12:53:22.249 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:22.249 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836499 2026-02-19T12:53:22.249 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2026-02-19T12:53:22.249 INFO:tasks.workunit.client.0.trial144.stderr:150976: waiting osd.0 seq 21474836499 2026-02-19T12:53:22.249 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:22.249 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836498 -lt 21474836499 2026-02-19T12:53:22.249 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836500 -lt 21474836499 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-270582939650 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-270582939650 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=270582939650 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 270582939650' 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: waiting osd.1 seq 270582939650 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 270582939651 -lt 270582939650 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-270582939650 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-270582939650 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=270582939650 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 270582939650' 2026-02-19T12:53:22.250 INFO:tasks.workunit.client.0.trial144.stderr:150976: waiting osd.2 seq 270582939650 2026-02-19T12:53:22.251 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:53:22.251 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 270582939651 -lt 270582939650 2026-02-19T12:53:22.251 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:226ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836500 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836500 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836500' 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: waiting osd.0 seq 21474836500 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836498 -lt 21474836500 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836500 -lt 21474836500 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-270582939651 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-270582939651 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:22.272 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=270582939651 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 270582939651' 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: waiting osd.1 seq 270582939651 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 270582939651 -lt 270582939651 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-270582939651 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-270582939651 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=270582939651 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 270582939651' 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: waiting osd.2 seq 270582939651 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 270582939651 -lt 270582939651 2026-02-19T12:53:22.273 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:22.953 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh7: flush_pg_stats: echo 3-244813135876 2026-02-19T12:53:22.953 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:22.953 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:53:22.953 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-244813135876 2026-02-19T12:53:22.953 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:22.953 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=244813135876 2026-02-19T12:53:22.953 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 244813135876' 2026-02-19T12:53:22.953 INFO:tasks.workunit.client.0.trial144.stderr:150976: waiting osd.3 seq 244813135876 2026-02-19T12:53:22.954 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:53:22.954 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 244813135877 -lt 244813135876 2026-02-19T12:53:22.954 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:53:22.954 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:22.954 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:22.954 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:53:22.954 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:22.954 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:22.954 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:22.954 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:22.954 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:22.954 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:22.955 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:22.955 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:53:22.955 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:22.955 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:22.955 INFO:tasks.workunit.client.0.trial144.stderr:150976: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:22.955 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:53:22.955 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:53:22.955 INFO:tasks.workunit.client.0.trial144.stderr:150976: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:53:22.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:53:22.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:53:22.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 150975 2026-02-19T12:53:22.985 INFO:tasks.workunit.client.0.trial144.stderr::2267: flush_pg_stats: echo 3-244813135877 2026-02-19T12:53:22.985 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:22.985 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:53:22.985 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-244813135877 2026-02-19T12:53:22.985 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:22.985 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=244813135877 2026-02-19T12:53:22.985 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 244813135877' 2026-02-19T12:53:22.985 INFO:tasks.workunit.client.0.trial144.stderr:150978: waiting osd.3 seq 244813135877 2026-02-19T12:53:22.986 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:53:22.986 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 244813135877 -lt 244813135877 2026-02-19T12:53:22.986 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:53:22.986 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:22.986 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:22.986 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:53:22.986 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:22.986 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:22.986 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:22.986 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:22.986 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:22.986 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:22.987 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:22.987 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:53:22.987 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:22.987 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:22.987 INFO:tasks.workunit.client.0.trial144.stderr:150978: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:22.987 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:53:22.987 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:53:22.987 INFO:tasks.workunit.client.0.trial144.stderr:150978: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:53:22.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:53:22.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:53:22.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:53:22.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:53:22.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:257: corrupt_and_repair_two: return_code=0 2026-02-19T12:53:22.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:258: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:53:22.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-19T12:53:22.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:53:22.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:53:22.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:53:22.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:53:23.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: local pg=2.0 2026-02-19T12:53:23.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:264: corrupt_and_repair_two: repair 2.0 2026-02-19T12:53:23.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:53:23.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:53:23.148 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:23.148 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:23.148 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:23.148 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:23.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:53:06.875410+0000 2026-02-19T12:53:23.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:53:23.457 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:53:23.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:53:06.875410+0000 2026-02-19T12:53:23.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:53:23.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:53:06.875410+0000 2026-02-19T12:53:23.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:53:23.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:53:23.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:23.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:23.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:23.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:23.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:23.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:23.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:53:06.875410+0000 '>' 2026-02-19T12:53:06.875410+0000 2026-02-19T12:53:23.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:53:24.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:53:24.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:24.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:24.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:24.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:24.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:24.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:24.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:53:06.875410+0000 '>' 2026-02-19T12:53:06.875410+0000 2026-02-19T12:53:24.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:53:25.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:53:25.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:25.788 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:25.788 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:25.788 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:25.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:25.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:25.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:53:06.875410+0000 '>' 2026-02-19T12:53:06.875410+0000 2026-02-19T12:53:25.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:53:26.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:53:26.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:26.946 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:26.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:26.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:26.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:26.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:27.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:53:06.875410+0000 '>' 2026-02-19T12:53:06.875410+0000 2026-02-19T12:53:27.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:53:28.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:53:28.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:28.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:28.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:28.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:28.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:28.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:28.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:53:06.875410+0000 '>' 2026-02-19T12:53:06.875410+0000 2026-02-19T12:53:28.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:53:29.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:53:29.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:29.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:29.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:29.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:29.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:29.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:29.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:53:06.875410+0000 '>' 2026-02-19T12:53:06.875410+0000 2026-02-19T12:53:29.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:53:30.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:53:30.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:30.418 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:30.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:30.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:30.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:30.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:30.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:53:24.306957+0000 '>' 2026-02-19T12:53:06.875410+0000 2026-02-19T12:53:30.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:53:30.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:268: corrupt_and_repair_two: pids= 2026-02-19T12:53:30.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:269: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:53:30.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:53:30.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:53:30.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 153835"' 2026-02-19T12:53:30.577 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 153835' 2026-02-19T12:53:30.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:270: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-19T12:53:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:53:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:53:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 153837"' 2026-02-19T12:53:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:53:30.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 153837' 2026-02-19T12:53:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:271: corrupt_and_repair_two: wait_background pids 2026-02-19T12:53:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 153835 153837' 2026-02-19T12:53:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:53:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:53:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 153835 2026-02-19T12:53:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/153838: /' 2026-02-19T12:53:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-19T12:53:30.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/153841: /' 2026-02-19T12:53:31.082 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:53:31.082 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:53:31.082 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:53:31.082 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:53:31.082 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:53:31.082 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:53:31.082 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:53:31.082 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:53:31.082 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:53:31.082 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:53:31.082 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: _ 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: hinfo_key 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: snapset 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:53:31.083 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:53:31.084 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T12:53:31.084 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:53:31.084 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:53:31.084 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING list-attrs 2026-02-19T12:53:31.084 INFO:tasks.workunit.client.0.trial144.stderr:153841: _ 2026-02-19T12:53:31.084 INFO:tasks.workunit.client.0.trial144.stderr:153841: hinfo_key 2026-02-19T12:53:31.084 INFO:tasks.workunit.client.0.trial144.stderr:153841: snapset 2026-02-19T12:53:31.084 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-19T12:53:31.084 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:53:31.084 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:53:31.084 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T12:53:31.084 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:53:31.084 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:53:31.084 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:53:31.084 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: start osd.2 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recos.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:53:31.086 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:53:31.087 INFO:tasks.workunit.client.0.trial144.stderr:153838: start osd.1 2026-02-19T12:53:32.572 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:53:32.572 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:53:32.573 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:53:32.573 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:53:32.573 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:53:32.573 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:53:32.573 INFO:tasks.workunit.client.0.trial144.stderr:153838: 2026-02-19T12:53:31.101+0000 7f19d4c3ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:32.573 INFO:tasks.workunit.client.0.trial144.stderr:153838: 2026-02-19T12:53:31.101+0000 7f19d4c3ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:32.573 INFO:tasks.workunit.client.0.trial144.stderr:153838: 2026-02-19T12:53:31.101+0000 7f19d4c3ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:32.573 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:53:32.573 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:53:32.573 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:53:32.574 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:53:32.574 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:53:32.574 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:32.574 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:53:32.574 INFO:tasks.workunit.client.0.trial144.stderr:153838: 0 2026-02-19T12:53:32.574 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:32.574 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:53:32.574 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:53:32.574 INFO:tasks.workunit.client.0.trial144.stderr:153838: 2026-02-19T12:53:31.557+0000 7f19d4c3ca40 -1 Falling back to public interface 2026-02-19T12:53:32.574 INFO:tasks.workunit.client.0.trial144.stderr:153838: 2026-02-19T12:53:32.221+0000 7f19d4c3ca40 -1 osd.1 64 log_to_monitors true 2026-02-19T12:53:32.574 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:53:32.574 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:32.574 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:53:32.575 INFO:tasks.workunit.client.0.trial144.stderr:153838: 1 2026-02-19T12:53:32.575 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:32.575 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:53:32.575 INFO:tasks.workunit.client.0.trial144.stderr:153838: osd.1 up in weight 1 up_from 68 up_thru 68 down_at 65 last_clean_interval [63,64) [v2:127.0.0.1:6810/2767720940,v1:127.0.0.1:6811/2767720940] [v2:127.0.0.1:6812/2767720940,v1:127.0.0.1:6813/2767720940] exists,up d92ac76d-154c-47e9-851c-f60a9fbd2c0f 2026-02-19T12:53:32.575 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:53:32.575 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:53:32.575 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:53:32.575 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:53:32.575 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:53:32.575 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:53:32.575 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:53:32.576 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:53:32.576 INFO:tasks.workunit.client.0.trial144.stderr:153838: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:53:32.576 INFO:tasks.workunit.client.0.trial144.stderr:153838: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.very_ops 2026-02-19T12:53:32.576 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-19T12:53:32.576 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T12:53:32.576 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:53:32.577 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:53:32.577 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:53:32.577 INFO:tasks.workunit.client.0.trial144.stderr:153841: 2026-02-19T12:53:31.097+0000 7fc2a2b5aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:32.577 INFO:tasks.workunit.client.0.trial144.stderr:153841: 2026-02-19T12:53:31.101+0000 7fc2a2b5aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:32.577 INFO:tasks.workunit.client.0.trial144.stderr:153841: 2026-02-19T12:53:31.101+0000 7fc2a2b5aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:32.577 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T12:53:32.577 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:53:32.577 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:53:32.577 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:53:32.577 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:53:32.577 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:32.578 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:53:32.578 INFO:tasks.workunit.client.0.trial144.stderr:153841: 0 2026-02-19T12:53:32.578 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:32.578 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:53:32.578 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:53:32.578 INFO:tasks.workunit.client.0.trial144.stderr:153841: 2026-02-19T12:53:31.565+0000 7fc2a2b5aa40 -1 Falling back to public interface 2026-02-19T12:53:32.578 INFO:tasks.workunit.client.0.trial144.stderr:153841: 2026-02-19T12:53:32.213+0000 7fc2a2b5aa40 -1 osd.2 64 log_to_monitors true 2026-02-19T12:53:32.578 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:53:32.578 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:32.578 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:53:32.578 INFO:tasks.workunit.client.0.trial144.stderr:153841: 1 2026-02-19T12:53:32.578 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:32.579 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:53:32.579 INFO:tasks.workunit.client.0.trial144.stderr:153841: osd.2 up in weight 1 up_from 68 up_thru 48 down_at 65 last_clean_interval [63,64) [v2:127.0.0.1:6818/3147191491,v1:127.0.0.1:6819/3147191491] [v2:127.0.0.1:6820/3147191491,v1:127.0.0.1:6821/3147191491] exists,up 4067a8a2-42ac-43fe-abfc-28e1d45c3c9e 2026-02-19T12:53:32.579 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:53:32.579 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:53:32.579 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:53:32.579 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:53:32.579 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:53:32.579 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:53:32.579 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:53:32.579 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:53:32.579 INFO:tasks.workunit.client.0.trial144.stderr:153841: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:53:33.128 INFO:tasks.workunit.client.0.trial144.stderr:153841: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:53:33.128 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:53:33.128 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:53:33.128 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:53:33.128 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:53:33.129 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:53:33.129 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:53:33.129 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:53:33.129 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:53:33.129 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:53:33.129 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:53:33.129 INFO:tasks.workunit.client.0.trial144.stderr:153838: 1 2026-02-19T12:53:33.129 INFO:tasks.workunit.client.0.trial144.stderr:153838: 2 2026-02-19T12:53:33.129 INFO:tasks.workunit.client.0.trial144.stderr:153838: 3' 2026-02-19T12:53:33.129 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:53:33.130 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:33.130 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:53:33.130 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836504 2026-02-19T12:53:33.130 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836504 2026-02-19T12:53:33.130 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504' 2026-02-19T12:53:33.130 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:33.130 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:53:33.130 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776130 2026-02-19T12:53:33.130 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776130 2026-02-19T12:53:33.130 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-292057776130' 2026-02-19T12:53:33.130 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:33.131 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:53:33.131 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776130 2026-02-19T12:53:33.131 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776130 2026-02-19T12:53:33.131 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-292057776130 2-292057776130' 2026-02-19T12:53:33.131 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:33.131 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:53:33.131 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=244813135880 2026-02-19T12:53:33.131 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 244813135880 2026-02-19T12:53:33.131 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-292057776130 2-292057776130 3-244813135880' 2026-02-19T12:53:33.131 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:33.131 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836504 2026-02-19T12:53:33.135 INFO:tasks.workunit.client.0.trial144.stderr:15383sh:1593: get_timeout_delays: echo true 2026-02-19T12:53:33.135 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:53:33.135 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:53:33.135 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:53:33.136 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:53:33.136 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:53:33.136 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:53:33.136 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:53:33.136 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:53:33.136 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:53:33.136 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:53:33.136 INFO:tasks.workunit.client.0.trial144.stderr:153841: 1 2026-02-19T12:53:33.137 INFO:tasks.workunit.client.0.trial144.stderr:153841: 2 2026-02-19T12:53:33.137 INFO:tasks.workunit.client.0.trial144.stderr:153841: 3' 2026-02-19T12:53:33.137 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:53:33.137 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:33.137 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:53:33.137 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836505 2026-02-19T12:53:33.137 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836505 2026-02-19T12:53:33.137 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505' 2026-02-19T12:53:33.138 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:33.138 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:53:33.138 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776131 2026-02-19T12:53:33.138 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776131 2026-02-19T12:53:33.138 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-292057776131' 2026-02-19T12:53:33.138 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:33.138 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:53:33.138 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776131 2026-02-19T12:53:33.138 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776131 2026-02-19T12:53:33.138 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-292057776131 2-292057776131' 2026-02-19T12:53:33.139 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:33.139 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:53:33.139 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=244813135881 2026-02-19T12:53:33.139 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 244813135881 2026-02-19T12:53:33.139 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-292057776131 2-292057776131 3-244813135881' 2026-02-19T12:53:33.139 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:33.139 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836505 2026-02-19T12:53:34.789 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:34.789 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836505 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836505 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836505' 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: waiting osd.0 seq 21474836505 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836502 -lt 21474836505 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836505 -lt 21474836505 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-292057776131 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-292057776131 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776131 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 292057776131' 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: waiting osd.1 seq 292057776131 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776131 -lt 292057776131 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-292057776131 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-292057776131 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776131 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 292057776131' 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: waiting osd.2 seq 292057776131 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776131 -lt 292057776131 2026-02-19T12:53:34.790 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:34.791 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-8: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:34.791 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:53:34.791 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836504 2026-02-19T12:53:34.791 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:34.791 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836504 2026-02-19T12:53:34.791 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836504' 2026-02-19T12:53:34.791 INFO:tasks.workunit.client.0.trial144.stderr:153838: waiting osd.0 seq 21474836504 2026-02-19T12:53:34.791 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:34.791 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836502 -lt 21474836504 2026-02-19T12:53:34.791 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:53:34.791 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:53:34.791 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:34.791 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836505 -lt 21474836504 2026-02-19T12:53:34.792 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:34.792 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-292057776130 2026-02-19T12:53:34.792 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:34.792 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:53:34.792 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-292057776130 2026-02-19T12:53:34.792 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:34.792 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776130 2026-02-19T12:53:34.792 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 292057776130' 2026-02-19T12:53:34.792 INFO:tasks.workunit.client.0.trial144.stderr:153838: waiting osd.1 seq 292057776130 2026-02-19T12:53:34.792 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:53:34.792 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776131 -lt 292057776130 2026-02-19T12:53:34.792 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:34.792 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-292057776130 2026-02-19T12:53:34.793 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:34.793 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:53:34.793 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-292057776130 2026-02-19T12:53:34.793 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:34.793 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776130 2026-02-19T12:53:34.793 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 292057776130' 2026-02-19T12:53:34.793 INFO:tasks.workunit.client.0.trial144.stderr:153838: waiting osd.2 seq 292057776130 2026-02-19T12:53:34.793 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:53:34.793 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776131 -lt 292057776130 2026-02-19T12:53:34.793 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-244813135880 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-244813135880 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=244813135880 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 244813135880' 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: waiting osd.3 seq 244813135880 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 244813135882 -lt 244813135880 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=3 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 3 = 5 2026-02-19T12:53:35.613 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 3 '!=' -1 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=3 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: lohelpers.sh:2267: flush_pg_stats: echo 3-244813135881 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-244813135881 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=244813135881 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 244813135881' 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: waiting osd.3 seq 244813135881 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 244813135882 -lt 244813135881 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=3 2026-02-19T12:53:35.614 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:35.615 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:35.615 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:35.615 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 3 = 5 2026-02-19T12:53:35.615 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 3 '!=' -1 2026-02-19T12:53:35.615 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T12:53:35.615 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=3 2026-02-19T12:53:35.615 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:53:35.615 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T12:53:35.615 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:53:35.615 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:35.615 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:36.369 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:36.369 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:36.369 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:36.370 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:36.370 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:36.370 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=3 2026-02-19T12:53:36.370 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:36.370 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:36.370 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:36.370 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 3 = 5 2026-02-19T12:53:36.370 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 3 '!=' 3 2026-02-19T12:53:36.371 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:53:36.371 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:53:36.371 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:53:36.371 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:53:36.371 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:53:36.371 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:53:36.371 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:53:36.371 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:53:36.372 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:53:36.372 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T12:53:36.372 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:53:36.372 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T12:53:36.372 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:53:36.372 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:36.372 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:36.372 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:36.373 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:36.373 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:36.373 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:36.382 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: getcal expression 2026-02-19T12:53:36.382 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:36.383 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:36.383 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:36.383 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:36.383 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=3 2026-02-19T12:53:36.383 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:36.383 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:36.383 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:36.383 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 3 = 5 2026-02-19T12:53:36.384 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 3 '!=' 3 2026-02-19T12:53:36.384 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:53:36.384 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:53:36.384 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:53:36.384 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:53:36.384 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:53:36.384 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:53:36.384 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:53:36.385 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:53:36.385 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:53:36.385 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T12:53:36.385 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:53:36.385 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T12:53:36.385 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:53:36.385 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:36.385 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:36.386 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:36.386 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:36.386 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:36.386 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:37.469 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:37.469 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=3 2026-02-19T12:53:37.469 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:37.469 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:37.469 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:37.469 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 3 = 5 2026-02-19T12:53:37.469 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 3 '!=' 3 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=3 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:37.470 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ce_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=3 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 3 = 5 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 3 '!=' 3 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=3 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:37.500 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:39.021 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:39.021 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 3 = 5 2026-02-19T12:53:39.021 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 3 '!=' 3 2026-02-19T12:53:39.021 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:53:39.021 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:53:39.021 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:53:39.022 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:53:39.022 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:53:39.022 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:53:39.022 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:53:39.022 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:53:39.022 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:53:39.022 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 13 )) 2026-02-19T12:53:39.022 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:53:39.022 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-19T12:53:39.022 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:53:39.022 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:39.023 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:39.023 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:39.023 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:39.023 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:39.023 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:39.023 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:39.023 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:53:39.023 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:39.023 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:39.023 INFO:tasks.workunit.client.0.trial144.stderr:153838: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:39.024 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:53:39.024 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:53:39.024 INFO:tasks.workunit.client.0.trial144.stderr:153838: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:53:39.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:53:39.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:53:39.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 153837 2026-02-19T12:53:39.029 INFO:tasks.workunit.client.0.trial144.stderr:ph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:39.029 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 3 = 5 2026-02-19T12:53:39.029 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 3 '!=' 3 2026-02-19T12:53:39.029 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:53:39.029 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:53:39.029 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:53:39.029 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:53:39.029 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:53:39.030 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:53:39.030 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:53:39.030 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:53:39.030 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:53:39.030 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 13 )) 2026-02-19T12:53:39.030 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:53:39.030 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-19T12:53:39.030 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:53:39.030 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:39.031 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:39.031 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:39.031 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:39.031 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:39.031 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:39.031 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:39.031 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:53:39.031 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:39.031 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:39.031 INFO:tasks.workunit.client.0.trial144.stderr:153841: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:39.032 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:53:39.032 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:53:39.032 INFO:tasks.workunit.client.0.trial144.stderr:153841: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:53:39.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:53:39.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:53:39.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:53:39.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:53:39.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:272: corrupt_and_repair_two: return_code=0 2026-02-19T12:53:39.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:273: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:53:39.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:275: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:53:39.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:276: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:53:39.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:323: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 3 1 2026-02-19T12:53:39.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:245: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-19T12:53:39.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:246: corrupt_and_repair_two: local poolname=ecpool 2026-02-19T12:53:39.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:247: corrupt_and_repair_two: local first=3 2026-02-19T12:53:39.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_two: local second=1 2026-02-19T12:53:39.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_two: pids= 2026-02-19T12:53:39.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:53:39.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:53:39.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:53:39.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 156644"' 2026-02-19T12:53:39.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 156644' 2026-02-19T12:53:39.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:53:39.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:53:39.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:53:39.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:53:39.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 156646"' 2026-02-19T12:53:39.054 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 156646' 2026-02-19T12:53:39.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_two: wait_background pids 2026-02-19T12:53:39.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 156644 156646' 2026-02-19T12:53:39.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:53:39.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:53:39.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/156647: /' 2026-02-19T12:53:39.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 156644 2026-02-19T12:53:39.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:53:39.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/156650: /' 2026-02-19T12:53:39.674 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:53:39.675 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:53:39.678 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:53:39.678 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:53:39.678 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:53:39.678 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:53:39.678 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:53:39.678 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:53:39.678 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:53:39.678 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:53:39.678 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:53:39.678 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:53:39.678 INFO:tasks.workunit.client.0.trial144.stderr:156650: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:53:39.678 INFO:tasks.workunit.client.0.trial144.stderr:156650: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:39.678 INFO:tasks.workunit.client.0.trial144.stderr:156650: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:39.678 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:39.679 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:39.679 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:53:39.679 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:53:39.679 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:53:39.679 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:53:39.679 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:53:39.679 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:53:39.679 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:53:39.679 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:53:39.679 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:53:39.679 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:53:39.679 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:53:39.679 INFO:tasks.workunit.client.0.trial144.stderr:156650: start osd.1 2026-02-19T12:53:39.679 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:1156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: remove 0#2:eb822e21:::SOMETHING:head# 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:53:39.703 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: start osd.3 2026-02-19T12:53:39.706 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:53:41.159 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:53:41.160 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:53:41.160 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:53:41.160 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:53:41.160 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:53:41.160 INFO:tasks.workunit.client.0.trial144.stderr:156650: 2026-02-19T12:53:39.693+0000 7f71411f0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:41.160 INFO:tasks.workunit.client.0.trial144.stderr:156650: 2026-02-19T12:53:39.693+0000 7f71411f0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:41.160 INFO:tasks.workunit.client.0.trial144.stderr:156650: 2026-02-19T12:53:39.693+0000 7f71411f0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:41.160 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:53:41.160 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:53:41.160 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:53:41.161 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:53:41.161 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:53:41.161 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:41.161 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:53:41.161 INFO:tasks.workunit.client.0.trial144.stderr:156650: 0 2026-02-19T12:53:41.161 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:41.161 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:53:41.161 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:53:41.161 INFO:tasks.workunit.client.0.trial144.stderr:156650: 2026-02-19T12:53:40.173+0000 7f71411f0a40 -1 Falling back to public interface 2026-02-19T12:53:41.161 INFO:tasks.workunit.client.0.trial144.stderr:156650: 2026-02-19T12:53:40.909+0000 7f71411f0a40 -1 osd.1 69 log_to_monitors true 2026-02-19T12:53:41.161 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:53:41.161 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:41.162 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:53:41.162 INFO:tasks.workunit.client.0.trial144.stderr:156650: 1 2026-02-19T12:53:41.162 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:41.162 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:53:41.162 INFO:tasks.workunit.client.0.trial144.stderr:156650: osd.1 up in weight 1 up_from 74 up_thru 68 down_at 70 last_clean_interval [68,69) [v2:127.0.0.1:6826/3988609587,v1:127.0.0.1:6827/3988609587] [v2:127.0.0.1:6828/3988609587,v1:127.0.0.1:6829/3988609587] exists,up d92ac76d-154c-47e9-851c-f60a9fbd2c0f 2026-02-19T12:53:41.162 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:53:41.162 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:53:41.162 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:53:41.162 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:53:41.162 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:53:41.162 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:53:41.162 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:53:41.163 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:53:41.163 INFO:tasks.workunit.client.0.trial144.stderr:156650: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:53:41.192 INFO:tasks.workunit.client.0.trial144.stderr:156650: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: g56647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:53:41.192 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:53:41.192 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:53:41.192 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:53:41.192 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:53:41.192 INFO:tasks.workunit.client.0.trial144.stderr:156647: 2026-02-19T12:53:39.717+0000 7f4e44d69a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:41.193 INFO:tasks.workunit.client.0.trial144.stderr:156647: 2026-02-19T12:53:39.717+0000 7f4e44d69a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:41.193 INFO:tasks.workunit.client.0.trial144.stderr:156647: 2026-02-19T12:53:39.721+0000 7f4e44d69a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:41.193 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:53:41.193 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:53:41.193 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:53:41.193 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:53:41.193 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:53:41.193 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:41.193 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:53:41.193 INFO:tasks.workunit.client.0.trial144.stderr:156647: 0 2026-02-19T12:53:41.193 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:41.194 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:53:41.194 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:53:41.194 INFO:tasks.workunit.client.0.trial144.stderr:156647: 2026-02-19T12:53:40.149+0000 7f4e44d69a40 -1 Falling back to public interface 2026-02-19T12:53:41.194 INFO:tasks.workunit.client.0.trial144.stderr:156647: 2026-02-19T12:53:40.841+0000 7f4e44d69a40 -1 osd.3 70 log_to_monitors true 2026-02-19T12:53:41.194 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:53:41.194 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:41.194 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:53:41.194 INFO:tasks.workunit.client.0.trial144.stderr:156647: 1 2026-02-19T12:53:41.194 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:41.194 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:53:41.194 INFO:tasks.workunit.client.0.trial144.stderr:156647: osd.3 up in weight 1 up_from 73 up_thru 73 down_at 70 last_clean_interval [57,70) [v2:127.0.0.1:6810/2677044165,v1:127.0.0.1:6811/2677044165] [v2:127.0.0.1:6812/2677044165,v1:127.0.0.1:6813/2677044165] exists,up 5b32660b-4e9c-40ea-9379-e74a0c3a6b89 2026-02-19T12:53:41.194 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:53:41.195 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:53:41.195 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:53:41.195 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:53:41.195 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:53:41.195 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:53:41.195 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:53:41.195 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:53:41.195 INFO:tasks.workunit.client.0.trial144.stderr:156647: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:53:41.711 INFO:tasks.workunit.client.0.trial144.stderr:156647: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:53:41.712 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:53:41.712 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:53:41.712 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:53:41.712 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:53:41.712 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:53:41.712 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:53:41.712 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:53:41.712 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:53:41.713 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:53:41.713 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:53:41.713 INFO:tasks.workunit.client.0.trial144.stderr:156650: 1 2026-02-19T12:53:41.713 INFO:tasks.workunit.client.0.trial144.stderr:156650: 2 2026-02-19T12:53:41.713 INFO:tasks.workunit.client.0.trial144.stderr:156650: 3' 2026-02-19T12:53:41.713 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:53:41.713 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:41.713 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:53:41.713 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836507 2026-02-19T12:53:41.713 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836507 2026-02-19T12:53:41.713 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507' 2026-02-19T12:53:41.713 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:41.714 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:53:41.714 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579906 2026-02-19T12:53:41.714 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579906 2026-02-19T12:53:41.714 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-317827579906' 2026-02-19T12:53:41.714 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:41.714 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:53:41.714 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776133 2026-02-19T12:53:41.714 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776133 2026-02-19T12:53:41.714 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-317827579906 2-292057776133' 2026-02-19T12:53:41.714 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:41.714 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:53:41.714 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=313532612610 2026-02-19T12:53:41.715 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 313532612610 2026-02-19T12:53:41.715 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-317827579906 2-292057776133 3-313532612610' 2026-02-19T12:53:41.715 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:41.715 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836507 2026-02-19T12:53:41.747 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/et_timeout_delays: echo true 2026-02-19T12:53:41.747 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:53:41.747 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:53:41.747 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:53:41.747 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:53:41.747 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:53:41.747 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:53:41.747 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:53:41.747 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:53:41.747 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:53:41.748 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:53:41.748 INFO:tasks.workunit.client.0.trial144.stderr:156647: 1 2026-02-19T12:53:41.748 INFO:tasks.workunit.client.0.trial144.stderr:156647: 2 2026-02-19T12:53:41.748 INFO:tasks.workunit.client.0.trial144.stderr:156647: 3' 2026-02-19T12:53:41.748 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:53:41.748 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:41.748 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:53:41.748 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836508 2026-02-19T12:53:41.748 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836508 2026-02-19T12:53:41.748 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836508' 2026-02-19T12:53:41.748 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:41.748 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:53:41.749 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579907 2026-02-19T12:53:41.749 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579907 2026-02-19T12:53:41.749 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836508 1-317827579907' 2026-02-19T12:53:41.749 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:41.749 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:53:41.749 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776134 2026-02-19T12:53:41.749 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776134 2026-02-19T12:53:41.749 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836508 1-317827579907 2-292057776134' 2026-02-19T12:53:41.749 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:41.749 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:53:41.749 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=313532612611 2026-02-19T12:53:41.749 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 313532612611 2026-02-19T12:53:41.749 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836508 1-317827579907 2-292057776134 3-313532612611' 2026-02-19T12:53:41.750 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:41.750 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836508 2026-02-19T12:53:43.368 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:43.369 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:53:43.369 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836507 2026-02-19T12:53:43.369 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:43.369 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836507 2026-02-19T12:53:43.369 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836507' 2026-02-19T12:53:43.369 INFO:tasks.workunit.client.0.trial144.stderr:156650: waiting osd.0 seq 21474836507 2026-02-19T12:53:43.369 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:43.369 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836506 -lt 21474836507 2026-02-19T12:53:43.369 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:53:43.370 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:53:43.370 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:43.370 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836508 -lt 21474836507 2026-02-19T12:53:43.370 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:43.370 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-317827579906 2026-02-19T12:53:43.370 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:43.370 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:53:43.370 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-317827579906 2026-02-19T12:53:43.370 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:43.370 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=317827579906 2026-02-19T12:53:43.370 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 317827579906' 2026-02-19T12:53:43.370 INFO:tasks.workunit.client.0.trial144.stderr:156650: waiting osd.1 seq 317827579906 2026-02-19T12:53:43.371 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:53:43.371 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 317827579907 -lt 317827579906 2026-02-19T12:53:43.371 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:43.371 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-292057776133 2026-02-19T12:53:43.371 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:43.371 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:53:43.371 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-292057776133 2026-02-19T12:53:43.371 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:43.371 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776133 2026-02-19T12:53:43.371 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 292057776133' 2026-02-19T12:53:43.371 INFO:tasks.workunit.client.0.trial144.stderr:156650: waiting osd.2 seq 292057776133 2026-02-19T12:53:43.371 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:53:43.372 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776134 -lt 292057776133 2026-02-19T12:53:43.372 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:43.396 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.shubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:43.396 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:53:43.396 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836508 2026-02-19T12:53:43.396 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:43.396 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836508 2026-02-19T12:53:43.396 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836508' 2026-02-19T12:53:43.396 INFO:tasks.workunit.client.0.trial144.stderr:156647: waiting osd.0 seq 21474836508 2026-02-19T12:53:43.396 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:43.397 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836506 -lt 21474836508 2026-02-19T12:53:43.397 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:53:43.397 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:53:43.397 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:43.397 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836508 -lt 21474836508 2026-02-19T12:53:43.397 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:43.397 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-317827579907 2026-02-19T12:53:43.397 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:43.397 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:53:43.397 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-317827579907 2026-02-19T12:53:43.397 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:43.397 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=317827579907 2026-02-19T12:53:43.398 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 317827579907' 2026-02-19T12:53:43.398 INFO:tasks.workunit.client.0.trial144.stderr:156647: waiting osd.1 seq 317827579907 2026-02-19T12:53:43.398 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:53:43.398 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 317827579907 -lt 317827579907 2026-02-19T12:53:43.398 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:43.398 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-292057776134 2026-02-19T12:53:43.398 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:43.398 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:53:43.398 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-292057776134 2026-02-19T12:53:43.398 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:43.398 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776134 2026-02-19T12:53:43.398 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 292057776134' 2026-02-19T12:53:43.399 INFO:tasks.workunit.client.0.trial144.stderr:156647: waiting osd.2 seq 292057776134 2026-02-19T12:53:43.399 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:53:43.399 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776134 -lt 292057776134 2026-02-19T12:53:43.399 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:44.108 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-313532612610 2026-02-19T12:53:44.109 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:44.109 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:53:44.109 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-313532612610 2026-02-19T12:53:44.109 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:44.109 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=313532612610 2026-02-19T12:53:44.109 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 313532612610' 2026-02-19T12:53:44.109 INFO:tasks.workunit.client.0.trial144.stderr:156650: waiting osd.3 seq 313532612610 2026-02-19T12:53:44.109 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:53:44.110 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 313532612611 -lt 313532612610 2026-02-19T12:53:44.110 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:53:44.110 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:44.110 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:44.110 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:53:44.110 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:44.110 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:44.110 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:44.110 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:44.110 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:44.110 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:44.110 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:44.111 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:53:44.111 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:44.111 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:44.111 INFO:tasks.workunit.client.0.trial144.stderr:156650: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:44.111 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:53:44.111 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:53:44.111 INFO:tasks.workunit.client.0.trial144.stderr:156650: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:53:44.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:53:44.111 INFO:tasks.workunit.client.0.trial144.stderr::2267: flush_pg_stats: echo 3-313532612611 2026-02-19T12:53:44.111 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:44.111 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:53:44.112 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-313532612611 2026-02-19T12:53:44.112 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:44.112 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=313532612611 2026-02-19T12:53:44.112 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 313532612611' 2026-02-19T12:53:44.112 INFO:tasks.workunit.client.0.trial144.stderr:156647: waiting osd.3 seq 313532612611 2026-02-19T12:53:44.112 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:53:44.112 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 313532612611 -lt 313532612611 2026-02-19T12:53:44.112 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:53:44.112 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:44.112 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:44.112 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:53:44.112 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:44.113 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:44.113 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:44.113 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:44.113 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:44.113 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:44.113 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:44.113 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:53:44.113 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:44.113 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:44.113 INFO:tasks.workunit.client.0.trial144.stderr:156647: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:44.113 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:53:44.113 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:53:44.114 INFO:tasks.workunit.client.0.trial144.stderr:156647: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:53:44.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:53:44.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:53:44.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 156646 2026-02-19T12:53:44.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:53:44.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:53:44.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:53:44.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:257: corrupt_and_repair_two: return_code=0 2026-02-19T12:53:44.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:258: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:53:44.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-19T12:53:44.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:53:44.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:53:44.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:53:44.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:53:44.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: local pg=2.0 2026-02-19T12:53:44.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:264: corrupt_and_repair_two: repair 2.0 2026-02-19T12:53:44.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:53:44.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:53:44.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:44.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:44.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:44.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:44.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:53:24.306957+0000 2026-02-19T12:53:44.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:53:44.570 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:53:44.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:53:24.306957+0000 2026-02-19T12:53:44.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:53:44.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:53:24.306957+0000 2026-02-19T12:53:44.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:53:44.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:53:44.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:44.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:44.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:44.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:44.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:44.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:44.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:53:24.306957+0000 '>' 2026-02-19T12:53:24.306957+0000 2026-02-19T12:53:44.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:53:45.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:53:45.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:45.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:45.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:45.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:45.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:45.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:45.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:53:24.306957+0000 '>' 2026-02-19T12:53:24.306957+0000 2026-02-19T12:53:45.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:53:46.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:53:46.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:46.903 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:46.903 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:46.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:46.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:46.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:47.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:53:24.306957+0000 '>' 2026-02-19T12:53:24.306957+0000 2026-02-19T12:53:47.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:53:48.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:53:48.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:53:48.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:53:48.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:53:48.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:53:48.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:53:48.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:53:48.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:53:44.795898+0000 '>' 2026-02-19T12:53:24.306957+0000 2026-02-19T12:53:48.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:53:48.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:268: corrupt_and_repair_two: pids= 2026-02-19T12:53:48.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:269: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:53:48.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:53:48.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:53:48.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 159436"' 2026-02-19T12:53:48.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 159436' 2026-02-19T12:53:48.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:270: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:53:48.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:53:48.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:53:48.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 159437"' 2026-02-19T12:53:48.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 159437' 2026-02-19T12:53:48.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:271: corrupt_and_repair_two: wait_background pids 2026-02-19T12:53:48.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 159436 159437' 2026-02-19T12:53:48.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:53:48.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:53:48.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 159436 2026-02-19T12:53:48.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:53:48.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/159439: /' 2026-02-19T12:53:48.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:53:48.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/159441: /' 2026-02-19T12:53:48.714 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:53:48.714 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:53:48.714 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:53:48.715 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:53:48.715 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:53:48.715 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:53:48.715 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:53:48.715 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:53:48.715 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:53:48.715 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:53:48.715 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:53:48.715 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:53:48.715 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:53:48.715 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:53:48.715 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:53:48.716 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:53:48.716 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:53:48.716 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:53:48.716 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:53:48.716 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:53:48.716 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:53:48.716 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:53:48.716 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-19T12:53:48.716 INFO:tasks.workunit.client.0.trial144.stderr:159441: _ 2026-02-19T12:53:48.716 INFO:tasks.workunit.client.0.trial144.stderr:159441: hinfo_key 2026-02-19T12:53:48.716 INFO:tasks.workunit.client.0.trial144.stderr:159441: snapset 2026-02-19T12:53:48.716 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:53:48.717 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:53:48.717 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:53:48.717 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:53:48.717 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:53:48.717 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:53:48.717 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:53:48.717 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:53:48.718 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:53:48.718 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:53:48.718 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:53:48.718 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:53:48.718 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:53:48.718 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:53:48.718 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:53:48.718 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:53:48.718 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:53:48.718 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:53:48.719 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:53:48.719 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:53:48.719 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:53:48.719 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:53:48.719 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:53:48.719 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:53:48.719 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:53:48.719 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:53:48.719 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:53:48.719 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:53:48.719 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:53:48.719 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:53:48.720 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-19T12:53:48.720 INFO:tasks.workunit.client.0.trial144.stderr:159439: Error getting attr on : 2.0s0_head,0#-4:00000000:::deep_scrub_2.0s0:head#, (61) No data available 2026-02-19T12:53:48.720 INFO:tasks.workunit.client.0.trial144.stderr:159439: Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-19T12:53:48.720 INFO:tasks.workunit.client.0.trial144.stderr:159439: _ 2026-02-19T12:53:48.720 INFO:tasks.workunit.client.0.trial144.stderr:159439: hinfo_key 2026-02-19T12:53:48.720 INFO:tasks.workunit.client.0.trial144.stderr:159439: snapset 2026-02-19T12:53:48.720 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:53:48.720 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:53:48.720 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:53:48.720 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:53:48.720 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:53:48.720 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:53:48.721 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:53:48.721 INFO:tasks.workunit.client.0.trial144.stderr:15s.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:53:48.721 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:53:48.721 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:53:48.721 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:53:48.721 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:53:48.721 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:53:48.721 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:53:48.721 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:53:48.721 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:53:48.721 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:53:48.721 INFO:tasks.workunit.client.0.trial144.stderr:159441: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:53:48.721 INFO:tasks.workunit.client.0.trial144.stderr:159441: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:48.722 INFO:tasks.workunit.client.0.trial144.stderr:159441: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:48.722 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:48.722 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:48.722 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:53:48.722 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:53:48.722 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:53:48.722 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:53:48.722 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:53:48.722 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:53:48.722 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:53:48.722 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:53:48.722 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:53:48.722 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:53:48.723 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:53:48.723 INFO:tasks.workunit.client.0.trial144.stderr:159441: start osd.1 2026-02-19T12:53:48.723 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_reco9439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:53:48.723 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:53:48.723 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:53:48.723 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:53:48.723 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:53:48.724 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:53:48.724 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:53:48.724 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:53:48.724 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:53:48.724 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:53:48.724 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:53:48.724 INFO:tasks.workunit.client.0.trial144.stderr:159439: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:53:48.724 INFO:tasks.workunit.client.0.trial144.stderr:159439: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:48.724 INFO:tasks.workunit.client.0.trial144.stderr:159439: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:48.724 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:48.724 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:48.724 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:53:48.725 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:53:48.725 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:53:48.725 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:53:48.725 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:53:48.725 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:53:48.725 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:53:48.725 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:53:48.725 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:53:48.725 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:53:48.725 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:53:48.725 INFO:tasks.workunit.client.0.trial144.stderr:159439: start osd.3 2026-02-19T12:53:50.198 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.very_ops 2026-02-19T12:53:50.198 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:53:50.198 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:53:50.198 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:53:50.198 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:53:50.198 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:53:50.199 INFO:tasks.workunit.client.0.trial144.stderr:159441: 2026-02-19T12:53:48.729+0000 7feec8a3aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:50.199 INFO:tasks.workunit.client.0.trial144.stderr:159441: 2026-02-19T12:53:48.733+0000 7feec8a3aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:50.199 INFO:tasks.workunit.client.0.trial144.stderr:159441: 2026-02-19T12:53:48.733+0000 7feec8a3aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:50.199 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:53:50.199 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:53:50.199 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:53:50.199 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:53:50.199 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:53:50.199 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:50.199 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:53:50.199 INFO:tasks.workunit.client.0.trial144.stderr:159441: 0 2026-02-19T12:53:50.199 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:50.200 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:53:50.200 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:53:50.200 INFO:tasks.workunit.client.0.trial144.stderr:159441: 2026-02-19T12:53:49.173+0000 7feec8a3aa40 -1 Falling back to public interface 2026-02-19T12:53:50.200 INFO:tasks.workunit.client.0.trial144.stderr:159441: 2026-02-19T12:53:49.781+0000 7feec8a3aa40 -1 osd.1 75 log_to_monitors true 2026-02-19T12:53:50.200 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:53:50.200 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:50.200 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:53:50.200 INFO:tasks.workunit.client.0.trial144.stderr:159441: 1 2026-02-19T12:53:50.200 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:50.200 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:53:50.200 INFO:tasks.workunit.client.0.trial144.stderr:159441: osd.1 up in weight 1 up_from 79 up_thru 79 down_at 76 last_clean_interval [74,75) [v2:127.0.0.1:6826/3186814968,v1:127.0.0.1:6827/3186814968] [v2:127.0.0.1:6828/3186814968,v1:127.0.0.1:6829/3186814968] exists,up d92ac76d-154c-47e9-851c-f60a9fbd2c0f 2026-02-19T12:53:50.200 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:53:50.201 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:53:50.201 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:53:50.201 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:53:50.201 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:53:50.201 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:53:50.201 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:53:50.201 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:53:50.201 INFO:tasks.workunit.client.0.trial144.stderr:159441: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:53:50.204 INFO:tasks.workunit.client.0.trial144.stderr:159441: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:53:50.204 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:53:50.204 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:53:50.204 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:53:50.204 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:53:50.204 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:53:50.205 INFO:tasks.workunit.client.0.trial144.stderr:159439: 2026-02-19T12:53:48.729+0000 7fa72b615a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:50.205 INFO:tasks.workunit.client.0.trial144.stderr:159439: 2026-02-19T12:53:48.733+0000 7fa72b615a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:50.205 INFO:tasks.workunit.client.0.trial144.stderr:159439: 2026-02-19T12:53:48.733+0000 7fa72b615a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:50.205 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:53:50.205 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:53:50.205 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:53:50.205 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:53:50.205 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:53:50.205 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:50.205 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:53:50.205 INFO:tasks.workunit.client.0.trial144.stderr:159439: 0 2026-02-19T12:53:50.205 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:50.206 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:53:50.206 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:53:50.206 INFO:tasks.workunit.client.0.trial144.stderr:159439: 2026-02-19T12:53:49.165+0000 7fa72b615a40 -1 Falling back to public interface 2026-02-19T12:53:50.206 INFO:tasks.workunit.client.0.trial144.stderr:159439: 2026-02-19T12:53:49.837+0000 7fa72b615a40 -1 osd.3 75 log_to_monitors true 2026-02-19T12:53:50.206 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:53:50.206 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:50.206 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:53:50.206 INFO:tasks.workunit.client.0.trial144.stderr:159439: 1 2026-02-19T12:53:50.206 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:50.206 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:53:50.206 INFO:tasks.workunit.client.0.trial144.stderr:159439: osd.3 up in weight 1 up_from 80 up_thru 80 down_at 76 last_clean_interval [73,75) [v2:127.0.0.1:6810/453166132,v1:127.0.0.1:6811/453166132] [v2:127.0.0.1:6812/453166132,v1:127.0.0.1:6813/453166132] exists,up 5b32660b-4e9c-40ea-9379-e74a0c3a6b89 2026-02-19T12:53:50.206 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:53:50.207 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:53:50.207 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:53:50.207 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:53:50.207 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:53:50.207 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:53:50.207 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:53:50.755 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays sh:1593: get_timeout_delays: echo true 2026-02-19T12:53:50.755 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:53:50.755 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:53:50.755 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:53:50.755 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:53:50.755 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:53:50.755 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:53:50.755 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:53:50.756 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:53:50.756 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:53:50.756 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:53:50.756 INFO:tasks.workunit.client.0.trial144.stderr:159441: 1 2026-02-19T12:53:50.756 INFO:tasks.workunit.client.0.trial144.stderr:159441: 2 2026-02-19T12:53:50.756 INFO:tasks.workunit.client.0.trial144.stderr:159441: 3' 2026-02-19T12:53:50.756 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:53:50.756 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:50.756 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:53:50.756 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836511 2026-02-19T12:53:50.756 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836511 2026-02-19T12:53:50.757 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836511' 2026-02-19T12:53:50.757 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:50.757 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:53:50.757 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416386 2026-02-19T12:53:50.757 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416386 2026-02-19T12:53:50.757 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836511 1-339302416386' 2026-02-19T12:53:50.757 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:50.757 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:53:50.757 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776137 2026-02-19T12:53:50.757 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776137 2026-02-19T12:53:50.757 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836511 1-339302416386 2-292057776137' 2026-02-19T12:53:50.757 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:50.757 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:53:50.758 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=343597383682 2026-02-19T12:53:50.758 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 343597383682 2026-02-19T12:53:50.758 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836511 1-339302416386 2-292057776137 3-343597383682' 2026-02-19T12:53:50.758 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:50.758 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836511 2026-02-19T12:53:50.770 INFO:tasks.workunit.client.0.trial144.stderr:1594490 .1 2026-02-19T12:53:50.770 INFO:tasks.workunit.client.0.trial144.stderr:159439: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:53:50.770 INFO:tasks.workunit.client.0.trial144.stderr:159439: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:53:50.770 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:53:50.770 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:53:50.770 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:53:50.770 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:53:50.770 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:53:50.770 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:53:50.771 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:53:50.771 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:53:50.771 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:53:50.771 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:53:50.771 INFO:tasks.workunit.client.0.trial144.stderr:159439: 1 2026-02-19T12:53:50.771 INFO:tasks.workunit.client.0.trial144.stderr:159439: 2 2026-02-19T12:53:50.771 INFO:tasks.workunit.client.0.trial144.stderr:159439: 3' 2026-02-19T12:53:50.771 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:53:50.771 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:50.771 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:53:50.771 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836512 2026-02-19T12:53:50.771 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836512 2026-02-19T12:53:50.772 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836512' 2026-02-19T12:53:50.772 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:50.772 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:53:50.772 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416387 2026-02-19T12:53:50.772 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416387 2026-02-19T12:53:50.772 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836512 1-339302416387' 2026-02-19T12:53:50.772 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:50.772 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:53:50.772 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776138 2026-02-19T12:53:50.772 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776138 2026-02-19T12:53:50.772 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836512 1-339302416387 2-292057776138' 2026-02-19T12:53:50.772 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:53:50.773 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:53:50.773 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=343597383683 2026-02-19T12:53:50.773 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 343597383683 2026-02-19T12:53:50.773 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836512 1-339302416387 2-292057776138 3-343597383683' 2026-02-19T12:53:52.410 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clo1: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:52.410 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:53:52.410 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836511 2026-02-19T12:53:52.410 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:52.410 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836511 2026-02-19T12:53:52.410 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836511' 2026-02-19T12:53:52.410 INFO:tasks.workunit.client.0.trial144.stderr:159441: waiting osd.0 seq 21474836511 2026-02-19T12:53:52.410 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:52.410 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836510 -lt 21474836511 2026-02-19T12:53:52.410 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:53:52.410 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:53:52.410 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:52.410 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836512 -lt 21474836511 2026-02-19T12:53:52.410 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-339302416386 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-339302416386 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=339302416386 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 339302416386' 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: waiting osd.1 seq 339302416386 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 339302416387 -lt 339302416386 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-292057776137 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-292057776137 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776137 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 292057776137' 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: waiting osd.2 seq 292057776137 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776138 -lt 292057776137 2026-02-19T12:53:52.411 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:52.422 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-ne.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:52.422 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836512 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836512 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836512 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836512' 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: waiting osd.0 seq 21474836512 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836510 -lt 21474836512 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836512 -lt 21474836512 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-339302416387 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-339302416387 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=339302416387 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 339302416387' 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: waiting osd.1 seq 339302416387 2026-02-19T12:53:52.423 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:53:52.424 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 339302416387 -lt 339302416387 2026-02-19T12:53:52.424 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:52.424 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-292057776138 2026-02-19T12:53:52.424 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:52.424 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:53:52.424 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-292057776138 2026-02-19T12:53:52.424 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:52.424 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776138 2026-02-19T12:53:52.424 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 292057776138' 2026-02-19T12:53:52.424 INFO:tasks.workunit.client.0.trial144.stderr:159439: waiting osd.2 seq 292057776138 2026-02-19T12:53:52.424 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:53:53.112 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 29205helpers.sh:2267: flush_pg_stats: echo 3-343597383682 2026-02-19T12:53:53.112 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:53.113 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:53:53.113 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-343597383682 2026-02-19T12:53:53.113 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:53.113 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=343597383682 2026-02-19T12:53:53.113 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 343597383682' 2026-02-19T12:53:53.113 INFO:tasks.workunit.client.0.trial144.stderr:159441: waiting osd.3 seq 343597383682 2026-02-19T12:53:53.113 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:53:53.113 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 343597383683 -lt 343597383682 2026-02-19T12:53:53.113 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:53:53.113 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:53.113 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:53.113 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:53:53.114 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:53.114 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:53.114 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:53.114 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:53.114 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:53.114 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:53.114 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:53.114 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:53:53.114 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:53.114 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:53.114 INFO:tasks.workunit.client.0.trial144.stderr:159441: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:53.114 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:53:53.115 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:53:53.115 INFO:tasks.workunit.client.0.trial144.stderr:159441: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:53:53.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:53:53.146 INFO:tasks.workunit.client.0.trial144.stderr:7776138 -lt 292057776138 2026-02-19T12:53:53.146 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:53:53.146 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-343597383683 2026-02-19T12:53:53.146 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:53:53.146 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:53:53.146 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-343597383683 2026-02-19T12:53:53.147 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:53:53.147 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=343597383683 2026-02-19T12:53:53.147 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 343597383683' 2026-02-19T12:53:53.147 INFO:tasks.workunit.client.0.trial144.stderr:159439: waiting osd.3 seq 343597383683 2026-02-19T12:53:53.147 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:53:53.147 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 343597383683 -lt 343597383683 2026-02-19T12:53:53.147 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:53:53.147 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:53.147 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:53.147 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:53:53.147 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:53:53.147 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:53:53.148 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:53:53.148 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:53:53.148 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:53:53.148 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:53:53.148 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:53:53.148 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:53:53.148 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:53:53.148 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:53:53.148 INFO:tasks.workunit.client.0.trial144.stderr:159439: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:53:53.148 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:53:53.148 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:53:53.148 INFO:tasks.workunit.client.0.trial144.stderr:159439: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:53:53.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:53:53.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:53:53.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 159437 2026-02-19T12:53:53.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:53:53.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:53:53.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:53:53.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:272: corrupt_and_repair_two: return_code=0 2026-02-19T12:53:53.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:273: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:53:53.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:275: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:53:53.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:276: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:53:53.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T12:53:53.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:53:53.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:53:53.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:53:53.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:53:53.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:53:53.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:53:53.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:53:53.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:53:53.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:53:53.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:53:53.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:53:53.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:53:53.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:53:53.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:53:53.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:53:53.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:53:53.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:53:53.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:53:53.285 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:53:53.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:53:53.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:53:53.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:53:53.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:53:53.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:53.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:53.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:53:53.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:53:53.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:53:53.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T12:53:53.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T12:53:53.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T12:53:53.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T12:53:53.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:53:53.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:53:53.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:53:53.318 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:53:53.318 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:53:53.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:53:53.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:53:53.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:53:53.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:53:53.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:53:53.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:53:53.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:53:53.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:53:53.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:53:53.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:53:53.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:53:53.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:53:53.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:53:53.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:53:53.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:53:53.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:53:53.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:53:53.329 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:53:53.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:53.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:53.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:53:53.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:53:53.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:53:53.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T12:53:53.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:53:53.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:53.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:53.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T12:53:53.335 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:53:53.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T12:53:53.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:53:53.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T12:53:53.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_and_repair_lrc_appends td/osd-scrub-repair 2026-02-19T12:53:53.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:901: TEST_corrupt_and_repair_lrc_appends: corrupt_and_repair_lrc td/osd-scrub-repair false 2026-02-19T12:53:53.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:884: corrupt_and_repair_lrc: local dir=td/osd-scrub-repair 2026-02-19T12:53:53.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:885: corrupt_and_repair_lrc: local allow_overwrites=false 2026-02-19T12:53:53.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:886: corrupt_and_repair_lrc: local poolname=ecpool 2026-02-19T12:53:53.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:888: corrupt_and_repair_lrc: run_mon td/osd-scrub-repair a 2026-02-19T12:53:53.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T12:53:53.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T12:53:53.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T12:53:53.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T12:53:53.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T12:53:53.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T12:53:53.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T12:53:53.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:53:53.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:53:53.359 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:53:53.359 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:53.359 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:53.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:53.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T12:53:53.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T12:53:53.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T12:53:53.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:53:53.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:53:53.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T12:53:53.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T12:53:53.393 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:53:53.393 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:53:53.394 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:53:53.394 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:53:53.394 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:53.394 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:53.394 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:53:53.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:53:53.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T12:53:53.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T12:53:53.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:53:53.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:53:53.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T12:53:53.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T12:53:53.452 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:53:53.452 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:53:53.452 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:53:53.452 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:53:53.452 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:53.453 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:53.453 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:53:53.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:53:53.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T12:53:53.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:889: corrupt_and_repair_lrc: run_mgr td/osd-scrub-repair x 2026-02-19T12:53:53.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T12:53:53.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T12:53:53.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T12:53:53.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T12:53:53.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T12:53:53.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T12:53:53.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T12:53:53.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:53:53.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:53:53.604 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:53:53.604 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:53.604 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:53.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:53.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:53:53.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:53:53.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: seq 0 9 2026-02-19T12:53:53.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:53:53.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 0 2026-02-19T12:53:53.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:53:53.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:53:53.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T12:53:53.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:53:53.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:53:53.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:53:53.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:53:53.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:53:53.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:53:53.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:53:53.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:53:53.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:53:53.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:53:53.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:53:53.628 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:53:53.628 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:53:53.628 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:53:53.629 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:53:53.629 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:53.629 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:53.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:53.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:53.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:53:53.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:53:53.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:53:53.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:53:53.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:53:53.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:53:53.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:53:53.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:53:53.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:53:53.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:53:53.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:53:53.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:53:53.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:53:53.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f2546a8b-2cf0-4800-a131-21acda0bcb1a 2026-02-19T12:53:53.631 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 f2546a8b-2cf0-4800-a131-21acda0bcb1a 2026-02-19T12:53:53.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f2546a8b-2cf0-4800-a131-21acda0bcb1a' 2026-02-19T12:53:53.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:53:53.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDhB5dpc9o5JhAAaUIxu1VoCXCIeVKvBkHPPw== 2026-02-19T12:53:53.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDhB5dpc9o5JhAAaUIxu1VoCXCIeVKvBkHPPw=="}' 2026-02-19T12:53:53.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f2546a8b-2cf0-4800-a131-21acda0bcb1a -i td/osd-scrub-repair/0/new.json 2026-02-19T12:53:53.729 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:53:53.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T12:53:53.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDhB5dpc9o5JhAAaUIxu1VoCXCIeVKvBkHPPw== --osd-uuid f2546a8b-2cf0-4800-a131-21acda0bcb1a 2026-02-19T12:53:53.755 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:53.752+0000 7fbf50138a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:53.757 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:53.752+0000 7fbf50138a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:53.758 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:53.752+0000 7fbf50138a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:53.758 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:53.756+0000 7fbf50138a40 -1 bdev(0x5621751a6c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:53:53.758 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:53.756+0000 7fbf50138a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T12:53:54.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T12:53:54.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:53:54.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T12:53:54.499 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:53:54.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:53:54.703 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:53:54.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T12:53:54.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:53:54.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:53:54.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:53:54.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:53:54.729 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:54.724+0000 7f2989d6ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:54.730 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:54.728+0000 7f2989d6ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:54.731 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:54.728+0000 7f2989d6ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:54.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T12:53:54.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:53:54.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:53:54.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:53:54.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:53:54.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:54.866 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:53:54.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:53:54.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:54.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:53:55.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:53:55.148 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:55.144+0000 7f2989d6ba40 -1 Falling back to public interface 2026-02-19T12:53:55.752 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:55.748+0000 7f2989d6ba40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:53:56.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:53:56.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:56.027 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:53:56.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:53:56.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:56.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:53:56.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:53:57.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:53:57.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:57.187 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:53:57.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:53:57.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:57.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:53:57.348 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1299586098,v1:127.0.0.1:6803/1299586098] [v2:127.0.0.1:6804/1299586098,v1:127.0.0.1:6805/1299586098] exists,up f2546a8b-2cf0-4800-a131-21acda0bcb1a 2026-02-19T12:53:57.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:53:57.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:53:57.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:53:57.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:53:57.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 1 2026-02-19T12:53:57.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:53:57.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:53:57.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T12:53:57.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:53:57.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:53:57.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:53:57.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:53:57.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:53:57.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:53:57.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:53:57.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:53:57.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:53:57.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:53:57.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:53:57.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:53:57.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:53:57.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:53:57.353 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:53:57.353 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:53:57.353 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:53:57.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:57.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:53:57.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:53:57.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:53:57.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:53:57.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:53:57.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:53:57.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:53:57.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:53:57.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:53:57.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:53:57.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:53:57.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:53:57.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:53:57.355 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:53:57.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=544b1644-d9ec-4a70-8bf5-c636c20f5f4c 2026-02-19T12:53:57.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 544b1644-d9ec-4a70-8bf5-c636c20f5f4c' 2026-02-19T12:53:57.356 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 544b1644-d9ec-4a70-8bf5-c636c20f5f4c 2026-02-19T12:53:57.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:53:57.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDlB5dpTN/XFRAAbEmbM9sqQKNnsEN2xviOCw== 2026-02-19T12:53:57.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDlB5dpTN/XFRAAbEmbM9sqQKNnsEN2xviOCw=="}' 2026-02-19T12:53:57.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 544b1644-d9ec-4a70-8bf5-c636c20f5f4c -i td/osd-scrub-repair/1/new.json 2026-02-19T12:53:57.530 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:53:57.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T12:53:57.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDlB5dpTN/XFRAAbEmbM9sqQKNnsEN2xviOCw== --osd-uuid 544b1644-d9ec-4a70-8bf5-c636c20f5f4c 2026-02-19T12:53:57.558 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:57.552+0000 7fe381b38a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:57.559 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:57.556+0000 7fe381b38a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:57.560 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:57.556+0000 7fe381b38a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:57.561 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:57.556+0000 7fe381b38a40 -1 bdev(0x55afafef2c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:53:57.561 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:57.556+0000 7fe381b38a40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T12:53:58.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T12:53:58.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:53:58.209 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:53:58.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T12:53:58.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:53:58.415 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:53:58.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T12:53:58.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:53:58.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:53:58.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:53:58.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:53:58.439 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:58.436+0000 7f91e93eca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:58.441 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:58.436+0000 7f91e93eca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:58.442 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:58.436+0000 7f91e93eca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:53:58.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T12:53:58.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:53:58.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:53:58.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:53:58.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:53:58.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:58.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:53:58.581 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:53:58.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:58.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:53:58.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:53:58.860 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:58.856+0000 7f91e93eca40 -1 Falling back to public interface 2026-02-19T12:53:59.520 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:53:59.516+0000 7f91e93eca40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:53:59.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:53:59.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:53:59.740 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:53:59.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:53:59.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:53:59.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:53:59.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:00.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:00.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:00.899 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:54:00.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:54:00.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:00.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:54:01.059 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/714115624,v1:127.0.0.1:6811/714115624] [v2:127.0.0.1:6812/714115624,v1:127.0.0.1:6813/714115624] exists,up 544b1644-d9ec-4a70-8bf5-c636c20f5f4c 2026-02-19T12:54:01.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:54:01.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:54:01.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:54:01.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 2 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:54:01.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:54:01.062 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:54:01.062 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:54:01.062 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:54:01.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:01.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:01.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:54:01.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:54:01.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:54:01.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:54:01.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:54:01.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:54:01.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:54:01.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:54:01.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:54:01.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:54:01.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:54:01.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:54:01.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:54:01.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=22c4c870-d88a-4d0f-bf1f-01c77f68ade0 2026-02-19T12:54:01.064 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 22c4c870-d88a-4d0f-bf1f-01c77f68ade0 2026-02-19T12:54:01.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 22c4c870-d88a-4d0f-bf1f-01c77f68ade0' 2026-02-19T12:54:01.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:54:01.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDpB5dpHqSiBBAAx5Ab57qmk33A07Z55VSibg== 2026-02-19T12:54:01.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDpB5dpHqSiBBAAx5Ab57qmk33A07Z55VSibg=="}' 2026-02-19T12:54:01.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 22c4c870-d88a-4d0f-bf1f-01c77f68ade0 -i td/osd-scrub-repair/2/new.json 2026-02-19T12:54:01.244 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:54:01.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T12:54:01.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDpB5dpHqSiBBAAx5Ab57qmk33A07Z55VSibg== --osd-uuid 22c4c870-d88a-4d0f-bf1f-01c77f68ade0 2026-02-19T12:54:01.272 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:01.268+0000 7f8818744a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:01.273 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:01.268+0000 7f8818744a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:01.274 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:01.268+0000 7f8818744a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:01.275 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:01.272+0000 7f8818744a40 -1 bdev(0x55658bde0c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T12:54:01.275 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:01.272+0000 7f8818744a40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T12:54:01.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T12:54:01.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:54:01.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T12:54:01.866 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T12:54:01.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:54:02.068 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T12:54:02.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T12:54:02.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:54:02.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:54:02.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:54:02.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:54:02.085 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:02.080+0000 7f16708c1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:02.086 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:02.084+0000 7f16708c1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:02.087 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:02.084+0000 7f16708c1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:02.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T12:54:02.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:54:02.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:54:02.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:54:02.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:54:02.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:02.230 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:54:02.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:54:02.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:02.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:54:02.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:02.464 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:02.460+0000 7f16708c1a40 -1 Falling back to public interface 2026-02-19T12:54:03.081 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:03.076+0000 7f16708c1a40 -1 osd.2 0 log_to_monitors true 2026-02-19T12:54:03.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:03.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:03.390 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:54:03.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:54:03.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:03.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:54:03.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:04.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:04.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:04.546 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:54:04.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:54:04.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:04.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:54:04.706 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2548858266,v1:127.0.0.1:6819/2548858266] [v2:127.0.0.1:6820/2548858266,v1:127.0.0.1:6821/2548858266] exists,up 22c4c870-d88a-4d0f-bf1f-01c77f68ade0 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 3 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:54:04.707 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:54:04.708 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:54:04.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:04.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:04.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:54:04.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:54:04.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:54:04.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:54:04.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:54:04.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:54:04.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:54:04.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:54:04.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:54:04.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:54:04.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:54:04.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:54:04.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:54:04.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=55da20d9-a5e9-4e5b-8bcc-8d8f03ecbf6d 2026-02-19T12:54:04.710 INFO:tasks.workunit.client.0.trial144.stdout:add osd3 55da20d9-a5e9-4e5b-8bcc-8d8f03ecbf6d 2026-02-19T12:54:04.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 55da20d9-a5e9-4e5b-8bcc-8d8f03ecbf6d' 2026-02-19T12:54:04.711 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:54:04.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDsB5dp6VE3KxAA0B4CT0u4phBUNF7n66K/6Q== 2026-02-19T12:54:04.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDsB5dp6VE3KxAA0B4CT0u4phBUNF7n66K/6Q=="}' 2026-02-19T12:54:04.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 55da20d9-a5e9-4e5b-8bcc-8d8f03ecbf6d -i td/osd-scrub-repair/3/new.json 2026-02-19T12:54:04.883 INFO:tasks.workunit.client.0.trial144.stdout:3 2026-02-19T12:54:04.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-19T12:54:04.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDsB5dp6VE3KxAA0B4CT0u4phBUNF7n66K/6Q== --osd-uuid 55da20d9-a5e9-4e5b-8bcc-8d8f03ecbf6d 2026-02-19T12:54:04.918 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:04.912+0000 7f6f9ac05a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:04.919 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:04.916+0000 7f6f9ac05a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:04.920 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:04.916+0000 7f6f9ac05a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:04.921 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:04.916+0000 7f6f9ac05a40 -1 bdev(0x55d4b3e68c00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-19T12:54:04.921 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:04.916+0000 7f6f9ac05a40 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-19T12:54:05.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-19T12:54:05.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:54:05.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T12:54:05.583 INFO:tasks.workunit.client.0.trial144.stdout:adding osd3 key to auth repository 2026-02-19T12:54:05.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:54:05.785 INFO:tasks.workunit.client.0.trial144.stdout:start osd.3 2026-02-19T12:54:05.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T12:54:05.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:54:05.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:54:05.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:54:05.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:54:05.810 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:05.804+0000 7f0ac7462a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:05.812 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:05.808+0000 7f0ac7462a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:05.812 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:05.808+0000 7f0ac7462a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:05.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T12:54:05.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:54:05.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:54:05.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:54:05.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:54:05.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:05.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:54:05.949 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:54:05.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:05.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:54:06.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:06.232 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:06.228+0000 7f0ac7462a40 -1 Falling back to public interface 2026-02-19T12:54:06.776 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:06.772+0000 7f0ac7462a40 -1 osd.3 0 log_to_monitors true 2026-02-19T12:54:07.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:07.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:07.107 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:54:07.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:54:07.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:07.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:54:07.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:08.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:08.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:08.267 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:54:08.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:54:08.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:08.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:54:08.425 INFO:tasks.workunit.client.0.trial144.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2536539005,v1:127.0.0.1:6827/2536539005] [v2:127.0.0.1:6828/2536539005,v1:127.0.0.1:6829/2536539005] exists,up 55da20d9-a5e9-4e5b-8bcc-8d8f03ecbf6d 2026-02-19T12:54:08.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:54:08.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:54:08.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:54:08.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:54:08.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 4 2026-02-19T12:54:08.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:54:08.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:54:08.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T12:54:08.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:54:08.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/4 2026-02-19T12:54:08.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:54:08.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:54:08.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:54:08.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:54:08.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/4' 2026-02-19T12:54:08.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/4/journal' 2026-02-19T12:54:08.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:54:08.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:54:08.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:54:08.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:54:08.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:54:08.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:54:08.430 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:54:08.430 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:54:08.430 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:54:08.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:08.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:08.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:54:08.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:54:08.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:54:08.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:54:08.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:54:08.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:54:08.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:54:08.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:54:08.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:54:08.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:54:08.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:54:08.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/4 2026-02-19T12:54:08.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:54:08.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5528b4bf-adb7-43e6-a5c8-fb54d6c96996 2026-02-19T12:54:08.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 5528b4bf-adb7-43e6-a5c8-fb54d6c96996' 2026-02-19T12:54:08.433 INFO:tasks.workunit.client.0.trial144.stdout:add osd4 5528b4bf-adb7-43e6-a5c8-fb54d6c96996 2026-02-19T12:54:08.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:54:08.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDwB5dppwZhGhAAg3WriKLBNqn4yllVwsnkdA== 2026-02-19T12:54:08.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDwB5dppwZhGhAAg3WriKLBNqn4yllVwsnkdA=="}' 2026-02-19T12:54:08.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5528b4bf-adb7-43e6-a5c8-fb54d6c96996 -i td/osd-scrub-repair/4/new.json 2026-02-19T12:54:08.603 INFO:tasks.workunit.client.0.trial144.stdout:4 2026-02-19T12:54:08.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/4/new.json 2026-02-19T12:54:08.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/4 --osd-journal=td/osd-scrub-repair/4/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDwB5dppwZhGhAAg3WriKLBNqn4yllVwsnkdA== --osd-uuid 5528b4bf-adb7-43e6-a5c8-fb54d6c96996 2026-02-19T12:54:08.630 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:08.628+0000 7f83d08eca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:08.632 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:08.628+0000 7f83d08eca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:08.633 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:08.628+0000 7f83d08eca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:08.634 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:08.628+0000 7f83d08eca40 -1 bdev(0x56496ff68c00 td/osd-scrub-repair/4/block) open stat got: (1) Operation not permitted 2026-02-19T12:54:08.634 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:08.628+0000 7f83d08eca40 -1 bluestore(td/osd-scrub-repair/4) _read_fsid unparsable uuid 2026-02-19T12:54:09.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/4/keyring 2026-02-19T12:54:09.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:54:09.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T12:54:09.281 INFO:tasks.workunit.client.0.trial144.stdout:adding osd4 key to auth repository 2026-02-19T12:54:09.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:54:09.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T12:54:09.484 INFO:tasks.workunit.client.0.trial144.stdout:start osd.4 2026-02-19T12:54:09.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/4 --osd-journal=td/osd-scrub-repair/4/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:54:09.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:54:09.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:54:09.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:54:09.508 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:09.504+0000 7ff421402a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:09.509 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:09.504+0000 7ff421402a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:09.510 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:09.508+0000 7ff421402a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:09.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T12:54:09.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:54:09.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T12:54:09.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:54:09.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:54:09.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:09.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:54:09.644 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:54:09.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:09.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T12:54:09.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:09.888 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:09.884+0000 7ff421402a40 -1 Falling back to public interface 2026-02-19T12:54:10.516 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:10.512+0000 7ff421402a40 -1 osd.4 0 log_to_monitors true 2026-02-19T12:54:10.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:10.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:10.799 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:54:10.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:54:10.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:10.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T12:54:10.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:11.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:11.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:11.961 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:54:11.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:54:11.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:11.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T12:54:12.118 INFO:tasks.workunit.client.0.trial144.stdout:osd.4 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2646339956,v1:127.0.0.1:6835/2646339956] [v2:127.0.0.1:6836/2646339956,v1:127.0.0.1:6837/2646339956] exists,up 5528b4bf-adb7-43e6-a5c8-fb54d6c96996 2026-02-19T12:54:12.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:54:12.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:54:12.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:54:12.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:54:12.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 5 2026-02-19T12:54:12.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:54:12.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:54:12.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-19T12:54:12.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:54:12.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:54:12.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:54:12.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:54:12.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:54:12.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:54:12.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-19T12:54:12.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-19T12:54:12.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:54:12.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:54:12.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:54:12.120 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:54:12.120 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:54:12.120 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:54:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/5 2026-02-19T12:54:12.122 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:54:12.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2df300a0-d24a-42d1-b116-112bdb34ad84 2026-02-19T12:54:12.123 INFO:tasks.workunit.client.0.trial144.stdout:add osd5 2df300a0-d24a-42d1-b116-112bdb34ad84 2026-02-19T12:54:12.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 2df300a0-d24a-42d1-b116-112bdb34ad84' 2026-02-19T12:54:12.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:54:12.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD0B5dpRyEvCBAAa/K8TLkoh9Zml16SIPvPJQ== 2026-02-19T12:54:12.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD0B5dpRyEvCBAAa/K8TLkoh9Zml16SIPvPJQ=="}' 2026-02-19T12:54:12.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2df300a0-d24a-42d1-b116-112bdb34ad84 -i td/osd-scrub-repair/5/new.json 2026-02-19T12:54:12.296 INFO:tasks.workunit.client.0.trial144.stdout:5 2026-02-19T12:54:12.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/5/new.json 2026-02-19T12:54:12.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD0B5dpRyEvCBAAa/K8TLkoh9Zml16SIPvPJQ== --osd-uuid 2df300a0-d24a-42d1-b116-112bdb34ad84 2026-02-19T12:54:12.324 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:12.320+0000 7fe54aa1ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:12.325 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:12.320+0000 7fe54aa1ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:12.326 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:12.320+0000 7fe54aa1ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:12.326 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:12.324+0000 7fe54aa1ea40 -1 bdev(0x55ad51faac00 td/osd-scrub-repair/5/block) open stat got: (1) Operation not permitted 2026-02-19T12:54:12.326 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:12.324+0000 7fe54aa1ea40 -1 bluestore(td/osd-scrub-repair/5) _read_fsid unparsable uuid 2026-02-19T12:54:12.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/5/keyring 2026-02-19T12:54:12.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:54:12.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-19T12:54:12.950 INFO:tasks.workunit.client.0.trial144.stdout:adding osd5 key to auth repository 2026-02-19T12:54:12.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:54:13.152 INFO:tasks.workunit.client.0.trial144.stdout:start osd.5 2026-02-19T12:54:13.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-19T12:54:13.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:54:13.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:54:13.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:54:13.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:54:13.177 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:13.172+0000 7f4e1d57da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:13.179 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:13.176+0000 7f4e1d57da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:13.180 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:13.176+0000 7f4e1d57da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:13.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-19T12:54:13.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:54:13.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T12:54:13.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:54:13.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:54:13.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:13.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:54:13.313 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:54:13.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:13.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:54:13.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:13.604 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:13.600+0000 7f4e1d57da40 -1 Falling back to public interface 2026-02-19T12:54:14.193 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:14.188+0000 7f4e1d57da40 -1 osd.5 0 log_to_monitors true 2026-02-19T12:54:14.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:14.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:14.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:54:14.471 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:54:14.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:14.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:54:14.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:15.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:15.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:15.633 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:54:15.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:54:15.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:15.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:54:15.788 INFO:tasks.workunit.client.0.trial144.stdout:osd.5 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1691294265,v1:127.0.0.1:6843/1691294265] [v2:127.0.0.1:6844/1691294265,v1:127.0.0.1:6845/1691294265] exists,up 2df300a0-d24a-42d1-b116-112bdb34ad84 2026-02-19T12:54:15.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:54:15.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:54:15.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:54:15.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:54:15.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 6 2026-02-19T12:54:15.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:54:15.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:54:15.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-19T12:54:15.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:54:15.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/6 2026-02-19T12:54:15.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:54:15.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:54:15.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:54:15.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:54:15.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/6' 2026-02-19T12:54:15.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/6/journal' 2026-02-19T12:54:15.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:54:15.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:54:15.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:54:15.794 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:54:15.794 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:54:15.794 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:54:15.794 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:54:15.794 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:54:15.794 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:54:15.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:15.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:15.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:54:15.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:54:15.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:54:15.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:54:15.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:54:15.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:54:15.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:54:15.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:54:15.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:54:15.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:54:15.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:54:15.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/6 2026-02-19T12:54:15.796 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:54:15.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ddf57f8d-01cd-4da7-82a4-755eeedc5700 2026-02-19T12:54:15.797 INFO:tasks.workunit.client.0.trial144.stdout:add osd6 ddf57f8d-01cd-4da7-82a4-755eeedc5700 2026-02-19T12:54:15.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 ddf57f8d-01cd-4da7-82a4-755eeedc5700' 2026-02-19T12:54:15.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:54:15.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD3B5dpSQ0MMBAAqTBtBvHzfBrGVsd+ALTyug== 2026-02-19T12:54:15.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD3B5dpSQ0MMBAAqTBtBvHzfBrGVsd+ALTyug=="}' 2026-02-19T12:54:15.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ddf57f8d-01cd-4da7-82a4-755eeedc5700 -i td/osd-scrub-repair/6/new.json 2026-02-19T12:54:15.968 INFO:tasks.workunit.client.0.trial144.stdout:6 2026-02-19T12:54:15.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/6/new.json 2026-02-19T12:54:15.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/6 --osd-journal=td/osd-scrub-repair/6/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD3B5dpSQ0MMBAAqTBtBvHzfBrGVsd+ALTyug== --osd-uuid ddf57f8d-01cd-4da7-82a4-755eeedc5700 2026-02-19T12:54:16.004 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:16.000+0000 7f5ddcd6ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:16.006 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:16.000+0000 7f5ddcd6ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:16.007 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:16.004+0000 7f5ddcd6ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:16.008 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:16.004+0000 7f5ddcd6ca40 -1 bdev(0x55ac6f2bac00 td/osd-scrub-repair/6/block) open stat got: (1) Operation not permitted 2026-02-19T12:54:16.008 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:16.004+0000 7f5ddcd6ca40 -1 bluestore(td/osd-scrub-repair/6) _read_fsid unparsable uuid 2026-02-19T12:54:16.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/6/keyring 2026-02-19T12:54:16.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:54:16.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-19T12:54:16.635 INFO:tasks.workunit.client.0.trial144.stdout:adding osd6 key to auth repository 2026-02-19T12:54:16.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:54:16.826 INFO:tasks.workunit.client.0.trial144.stdout:start osd.6 2026-02-19T12:54:16.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-19T12:54:16.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/6 --osd-journal=td/osd-scrub-repair/6/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:54:16.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:54:16.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:54:16.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:54:16.843 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:16.840+0000 7f01fcf2ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:16.844 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:16.840+0000 7f01fcf2ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:16.845 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:16.840+0000 7f01fcf2ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:16.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-19T12:54:16.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:54:16.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T12:54:16.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:54:16.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:54:16.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:16.990 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:54:16.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:54:16.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:16.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T12:54:17.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:17.260 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:17.256+0000 7f01fcf2ba40 -1 Falling back to public interface 2026-02-19T12:54:17.908 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:17.904+0000 7f01fcf2ba40 -1 osd.6 0 log_to_monitors true 2026-02-19T12:54:18.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:18.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:18.150 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:54:18.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:54:18.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:18.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T12:54:18.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:19.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:19.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:19.339 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:54:19.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:54:19.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:19.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T12:54:19.500 INFO:tasks.workunit.client.0.trial144.stdout:osd.6 up in weight 1 up_from 30 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3990277936,v1:127.0.0.1:6851/3990277936] [v2:127.0.0.1:6852/3990277936,v1:127.0.0.1:6853/3990277936] exists,up ddf57f8d-01cd-4da7-82a4-755eeedc5700 2026-02-19T12:54:19.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:54:19.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:54:19.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:54:19.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:54:19.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 7 2026-02-19T12:54:19.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:54:19.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:54:19.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-19T12:54:19.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:54:19.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/7 2026-02-19T12:54:19.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:54:19.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:54:19.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:54:19.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:54:19.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/7' 2026-02-19T12:54:19.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/7/journal' 2026-02-19T12:54:19.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:54:19.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:54:19.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:54:19.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:54:19.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:54:19.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:54:19.505 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:54:19.506 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:54:19.506 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:54:19.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:19.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:19.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:54:19.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:54:19.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:54:19.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:54:19.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:54:19.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:54:19.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:54:19.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:54:19.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:54:19.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:54:19.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:54:19.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/7 2026-02-19T12:54:19.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:54:19.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a56c50bc-5085-4c73-befa-67e98ae850c7 2026-02-19T12:54:19.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 a56c50bc-5085-4c73-befa-67e98ae850c7' 2026-02-19T12:54:19.508 INFO:tasks.workunit.client.0.trial144.stdout:add osd7 a56c50bc-5085-4c73-befa-67e98ae850c7 2026-02-19T12:54:19.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:54:19.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD7B5dpkbvmHhAABqKns4rrpN0MPURfxwEafQ== 2026-02-19T12:54:19.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD7B5dpkbvmHhAABqKns4rrpN0MPURfxwEafQ=="}' 2026-02-19T12:54:19.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a56c50bc-5085-4c73-befa-67e98ae850c7 -i td/osd-scrub-repair/7/new.json 2026-02-19T12:54:19.678 INFO:tasks.workunit.client.0.trial144.stdout:7 2026-02-19T12:54:19.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/7/new.json 2026-02-19T12:54:19.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/7 --osd-journal=td/osd-scrub-repair/7/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD7B5dpkbvmHhAABqKns4rrpN0MPURfxwEafQ== --osd-uuid a56c50bc-5085-4c73-befa-67e98ae850c7 2026-02-19T12:54:19.705 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:19.700+0000 7f3382361a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:19.707 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:19.704+0000 7f3382361a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:19.708 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:19.704+0000 7f3382361a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:19.708 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:19.704+0000 7f3382361a40 -1 bdev(0x558bb20c8c00 td/osd-scrub-repair/7/block) open stat got: (1) Operation not permitted 2026-02-19T12:54:19.708 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:19.704+0000 7f3382361a40 -1 bluestore(td/osd-scrub-repair/7) _read_fsid unparsable uuid 2026-02-19T12:54:20.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/7/keyring 2026-02-19T12:54:20.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:54:20.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-19T12:54:20.350 INFO:tasks.workunit.client.0.trial144.stdout:adding osd7 key to auth repository 2026-02-19T12:54:20.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:54:20.538 INFO:tasks.workunit.client.0.trial144.stdout:start osd.7 2026-02-19T12:54:20.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-19T12:54:20.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/7 --osd-journal=td/osd-scrub-repair/7/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:54:20.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:54:20.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:54:20.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:54:20.563 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:20.560+0000 7ff8e90aea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:20.564 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:20.560+0000 7ff8e90aea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:20.565 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:20.560+0000 7ff8e90aea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:20.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-19T12:54:20.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:54:20.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-19T12:54:20.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:54:20.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:54:20.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:20.702 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:54:20.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:54:20.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:20.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-19T12:54:20.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:20.980 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:20.976+0000 7ff8e90aea40 -1 Falling back to public interface 2026-02-19T12:54:21.612 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:21.608+0000 7ff8e90aea40 -1 osd.7 0 log_to_monitors true 2026-02-19T12:54:21.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:21.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:21.863 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:54:21.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:54:21.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:21.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-19T12:54:22.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:23.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:23.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:23.022 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:54:23.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:54:23.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:23.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-19T12:54:23.179 INFO:tasks.workunit.client.0.trial144.stdout:osd.7 up in weight 1 up_from 34 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/3571144498,v1:127.0.0.1:6859/3571144498] [v2:127.0.0.1:6860/3571144498,v1:127.0.0.1:6861/3571144498] exists,up a56c50bc-5085-4c73-befa-67e98ae850c7 2026-02-19T12:54:23.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:54:23.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:54:23.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:54:23.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:54:23.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 8 2026-02-19T12:54:23.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:54:23.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:54:23.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-19T12:54:23.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:54:23.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/8 2026-02-19T12:54:23.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:54:23.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:54:23.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:54:23.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:54:23.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/8' 2026-02-19T12:54:23.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/8/journal' 2026-02-19T12:54:23.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:54:23.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:54:23.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:54:23.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:54:23.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:54:23.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:54:23.184 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:54:23.184 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:54:23.184 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:54:23.185 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:23.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:23.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:54:23.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:54:23.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:54:23.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:54:23.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:54:23.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:54:23.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:54:23.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:54:23.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:54:23.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:54:23.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:54:23.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/8 2026-02-19T12:54:23.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:54:23.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d5b3ccc8-f04d-4ad9-ab0e-be00d4216548 2026-02-19T12:54:23.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 d5b3ccc8-f04d-4ad9-ab0e-be00d4216548' 2026-02-19T12:54:23.187 INFO:tasks.workunit.client.0.trial144.stdout:add osd8 d5b3ccc8-f04d-4ad9-ab0e-be00d4216548 2026-02-19T12:54:23.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:54:23.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD/B5dpWgPRCxAAkxcyD7RGGeO2qjCbuGvT6Q== 2026-02-19T12:54:23.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD/B5dpWgPRCxAAkxcyD7RGGeO2qjCbuGvT6Q=="}' 2026-02-19T12:54:23.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d5b3ccc8-f04d-4ad9-ab0e-be00d4216548 -i td/osd-scrub-repair/8/new.json 2026-02-19T12:54:23.358 INFO:tasks.workunit.client.0.trial144.stdout:8 2026-02-19T12:54:23.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/8/new.json 2026-02-19T12:54:23.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/8 --osd-journal=td/osd-scrub-repair/8/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD/B5dpWgPRCxAAkxcyD7RGGeO2qjCbuGvT6Q== --osd-uuid d5b3ccc8-f04d-4ad9-ab0e-be00d4216548 2026-02-19T12:54:23.384 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:23.384+0000 7f11cb87da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:23.385 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:23.384+0000 7f11cb87da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:23.386 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:23.384+0000 7f11cb87da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:23.387 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:23.384+0000 7f11cb87da40 -1 bdev(0x55742a3f8c00 td/osd-scrub-repair/8/block) open stat got: (1) Operation not permitted 2026-02-19T12:54:23.387 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:23.384+0000 7f11cb87da40 -1 bluestore(td/osd-scrub-repair/8) _read_fsid unparsable uuid 2026-02-19T12:54:24.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/8/keyring 2026-02-19T12:54:24.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:54:24.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-19T12:54:24.062 INFO:tasks.workunit.client.0.trial144.stdout:adding osd8 key to auth repository 2026-02-19T12:54:24.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:54:24.258 INFO:tasks.workunit.client.0.trial144.stdout:start osd.8 2026-02-19T12:54:24.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-19T12:54:24.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/8 --osd-journal=td/osd-scrub-repair/8/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:54:24.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:54:24.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:54:24.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:54:24.274 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:24.272+0000 7fbac5836a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:24.275 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:24.276+0000 7fbac5836a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:24.276 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:24.276+0000 7fbac5836a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:24.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-19T12:54:24.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:54:24.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-19T12:54:24.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:54:24.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:54:24.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:24.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:54:24.419 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:54:24.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:24.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-19T12:54:24.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:24.692 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:24.692+0000 7fbac5836a40 -1 Falling back to public interface 2026-02-19T12:54:25.309 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:25.308+0000 7fbac5836a40 -1 osd.8 0 log_to_monitors true 2026-02-19T12:54:25.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:25.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:25.575 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:54:25.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:54:25.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:25.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-19T12:54:25.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:26.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:26.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:26.738 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:54:26.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:54:26.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:26.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-19T12:54:26.890 INFO:tasks.workunit.client.0.trial144.stdout:osd.8 up in weight 1 up_from 38 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/2847709659,v1:127.0.0.1:6867/2847709659] [v2:127.0.0.1:6868/2847709659,v1:127.0.0.1:6869/2847709659] exists,up d5b3ccc8-f04d-4ad9-ab0e-be00d4216548 2026-02-19T12:54:26.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:54:26.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:54:26.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:54:26.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:54:26.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 9 2026-02-19T12:54:26.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:54:26.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:54:26.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=9 2026-02-19T12:54:26.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:54:26.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:54:26.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:54:26.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:54:26.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:54:26.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:54:26.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-19T12:54:26.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-19T12:54:26.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:54:26.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:54:26.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:54:26.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:54:26.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:54:26.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:54:26.896 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:54:26.896 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:54:26.896 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:54:26.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:26.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:26.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:54:26.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:54:26.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:54:26.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:54:26.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:54:26.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:54:26.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:54:26.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:54:26.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:54:26.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:54:26.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:54:26.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/9 2026-02-19T12:54:26.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:54:26.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=19beb6b7-cd3d-4c41-8734-6c7b68c1a39d 2026-02-19T12:54:26.898 INFO:tasks.workunit.client.0.trial144.stdout:add osd9 19beb6b7-cd3d-4c41-8734-6c7b68c1a39d 2026-02-19T12:54:26.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd9 19beb6b7-cd3d-4c41-8734-6c7b68c1a39d' 2026-02-19T12:54:26.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:54:26.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQACCJdpGqcpNhAAL6C/NgqabBgAdTyo1gjPvg== 2026-02-19T12:54:26.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQACCJdpGqcpNhAAL6C/NgqabBgAdTyo1gjPvg=="}' 2026-02-19T12:54:26.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 19beb6b7-cd3d-4c41-8734-6c7b68c1a39d -i td/osd-scrub-repair/9/new.json 2026-02-19T12:54:27.063 INFO:tasks.workunit.client.0.trial144.stdout:9 2026-02-19T12:54:27.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/9/new.json 2026-02-19T12:54:27.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 9 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQACCJdpGqcpNhAAL6C/NgqabBgAdTyo1gjPvg== --osd-uuid 19beb6b7-cd3d-4c41-8734-6c7b68c1a39d 2026-02-19T12:54:27.089 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:27.088+0000 7fa9058b7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:27.090 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:27.092+0000 7fa9058b7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:27.091 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:27.092+0000 7fa9058b7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:27.092 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:27.092+0000 7fa9058b7a40 -1 bdev(0x556fe516ac00 td/osd-scrub-repair/9/block) open stat got: (1) Operation not permitted 2026-02-19T12:54:27.092 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:27.092+0000 7fa9058b7a40 -1 bluestore(td/osd-scrub-repair/9) _read_fsid unparsable uuid 2026-02-19T12:54:27.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/9/keyring 2026-02-19T12:54:27.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:54:27.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd9 key to auth repository 2026-02-19T12:54:27.775 INFO:tasks.workunit.client.0.trial144.stdout:adding osd9 key to auth repository 2026-02-19T12:54:27.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:54:27.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.9 2026-02-19T12:54:27.974 INFO:tasks.workunit.client.0.trial144.stdout:start osd.9 2026-02-19T12:54:27.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 9 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:54:27.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:54:27.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:54:27.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:54:27.999 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:28.000+0000 7f6e44a06a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:28.001 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:28.000+0000 7f6e44a06a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:28.002 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:28.000+0000 7f6e44a06a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:28.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 9 2026-02-19T12:54:28.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:54:28.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-19T12:54:28.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:54:28.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:54:28.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:28.134 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:54:28.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:54:28.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:28.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:54:28.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:28.444 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:28.444+0000 7f6e44a06a40 -1 Falling back to public interface 2026-02-19T12:54:29.121 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:29.120+0000 7f6e44a06a40 -1 osd.9 0 log_to_monitors true 2026-02-19T12:54:29.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:29.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:29.289 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:54:29.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:54:29.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:29.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:54:29.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:30.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:30.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:30.449 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:54:30.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:54:30.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:30.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:54:30.600 INFO:tasks.workunit.client.0.trial144.stdout:osd.9 up in weight 1 up_from 42 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/2531943667,v1:127.0.0.1:6875/2531943667] [v2:127.0.0.1:6876/2531943667,v1:127.0.0.1:6877/2531943667] exists,up 19beb6b7-cd3d-4c41-8734-6c7b68c1a39d 2026-02-19T12:54:30.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:54:30.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:54:30.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:54:30.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:893: corrupt_and_repair_lrc: create_rbd_pool 2026-02-19T12:54:30.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T12:54:30.746 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T12:54:30.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T12:54:30.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T12:54:30.973 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T12:54:30.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:54:31.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T12:54:32.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:894: corrupt_and_repair_lrc: wait_for_clean 2026-02-19T12:54:32.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:54:32.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:54:32.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:54:32.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:54:32.265 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:54:32.265 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:54:32.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:54:32.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:54:32.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:54:32.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:54:32.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:54:32.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:54:32.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:54:32.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:54:32.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:54:32.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:54:32.534 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:54:32.534 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:54:32.534 INFO:tasks.workunit.client.0.trial144.stderr:3 2026-02-19T12:54:32.534 INFO:tasks.workunit.client.0.trial144.stderr:4 2026-02-19T12:54:32.535 INFO:tasks.workunit.client.0.trial144.stderr:5 2026-02-19T12:54:32.535 INFO:tasks.workunit.client.0.trial144.stderr:6 2026-02-19T12:54:32.535 INFO:tasks.workunit.client.0.trial144.stderr:7 2026-02-19T12:54:32.535 INFO:tasks.workunit.client.0.trial144.stderr:8 2026-02-19T12:54:32.535 INFO:tasks.workunit.client.0.trial144.stderr:9' 2026-02-19T12:54:32.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:54:32.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:32.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:54:32.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836489 2026-02-19T12:54:32.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836489 2026-02-19T12:54:32.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489' 2026-02-19T12:54:32.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:32.608 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:54:32.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705672 2026-02-19T12:54:32.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705672 2026-02-19T12:54:32.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-38654705672' 2026-02-19T12:54:32.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:32.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:54:32.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-19T12:54:32.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-19T12:54:32.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-38654705672 2-55834574855' 2026-02-19T12:54:32.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:32.754 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:54:32.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411334 2026-02-19T12:54:32.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411334 2026-02-19T12:54:32.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-38654705672 2-55834574855 3-77309411334' 2026-02-19T12:54:32.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:32.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:54:32.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280518 2026-02-19T12:54:32.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280518 2026-02-19T12:54:32.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-38654705672 2-55834574855 3-77309411334 4-94489280518' 2026-02-19T12:54:32.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:32.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:54:32.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149701 2026-02-19T12:54:32.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149701 2026-02-19T12:54:32.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-38654705672 2-55834574855 3-77309411334 4-94489280518 5-111669149701' 2026-02-19T12:54:32.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:32.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:54:33.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018884 2026-02-19T12:54:33.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018884 2026-02-19T12:54:33.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-38654705672 2-55834574855 3-77309411334 4-94489280518 5-111669149701 6-128849018884' 2026-02-19T12:54:33.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:33.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:54:33.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888068 2026-02-19T12:54:33.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888068 2026-02-19T12:54:33.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-38654705672 2-55834574855 3-77309411334 4-94489280518 5-111669149701 6-128849018884 7-146028888068' 2026-02-19T12:54:33.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:33.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:54:33.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757251 2026-02-19T12:54:33.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757251 2026-02-19T12:54:33.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-38654705672 2-55834574855 3-77309411334 4-94489280518 5-111669149701 6-128849018884 7-146028888068 8-163208757251' 2026-02-19T12:54:33.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:33.185 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:54:33.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626434 2026-02-19T12:54:33.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626434 2026-02-19T12:54:33.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-38654705672 2-55834574855 3-77309411334 4-94489280518 5-111669149701 6-128849018884 7-146028888068 8-163208757251 9-180388626434' 2026-02-19T12:54:33.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:33.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836489 2026-02-19T12:54:33.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:33.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:54:33.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836489 2026-02-19T12:54:33.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:33.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836489 2026-02-19T12:54:33.263 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836489 2026-02-19T12:54:33.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-19T12:54:33.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:54:33.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-19T12:54:33.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:54:34.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:54:34.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:54:34.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-19T12:54:34.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:34.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705672 2026-02-19T12:54:34.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:34.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:54:34.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705672 2026-02-19T12:54:34.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:34.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705672 2026-02-19T12:54:34.589 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705672 2026-02-19T12:54:34.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705672' 2026-02-19T12:54:34.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:54:34.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705672 -lt 38654705672 2026-02-19T12:54:34.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:34.763 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574855 2026-02-19T12:54:34.763 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:34.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:54:34.766 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574855 2026-02-19T12:54:34.766 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:34.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574855 2026-02-19T12:54:34.767 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574855 2026-02-19T12:54:34.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574855' 2026-02-19T12:54:34.768 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:54:34.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-19T12:54:34.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:34.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411334 2026-02-19T12:54:34.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:34.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:54:34.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-77309411334 2026-02-19T12:54:34.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:34.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411334 2026-02-19T12:54:34.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 77309411334' 2026-02-19T12:54:34.930 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.3 seq 77309411334 2026-02-19T12:54:34.930 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:54:35.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411335 -lt 77309411334 2026-02-19T12:54:35.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:35.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280518 2026-02-19T12:54:35.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:35.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:54:35.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-94489280518 2026-02-19T12:54:35.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:35.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280518 2026-02-19T12:54:35.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 94489280518' 2026-02-19T12:54:35.091 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.4 seq 94489280518 2026-02-19T12:54:35.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:54:35.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280518 -lt 94489280518 2026-02-19T12:54:35.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:35.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-111669149701 2026-02-19T12:54:35.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:35.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:54:35.249 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-111669149701 2026-02-19T12:54:35.249 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:35.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149701 2026-02-19T12:54:35.251 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.5 seq 111669149701 2026-02-19T12:54:35.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 111669149701' 2026-02-19T12:54:35.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:54:35.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149701 -lt 111669149701 2026-02-19T12:54:35.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:35.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-128849018884 2026-02-19T12:54:35.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:35.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:54:35.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-128849018884 2026-02-19T12:54:35.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:35.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018884 2026-02-19T12:54:35.415 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.6 seq 128849018884 2026-02-19T12:54:35.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 128849018884' 2026-02-19T12:54:35.416 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:54:35.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018885 -lt 128849018884 2026-02-19T12:54:35.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:35.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-146028888068 2026-02-19T12:54:35.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:35.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:54:35.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-146028888068 2026-02-19T12:54:35.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:35.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888068 2026-02-19T12:54:35.579 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.7 seq 146028888068 2026-02-19T12:54:35.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 146028888068' 2026-02-19T12:54:35.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:54:35.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888068 -lt 146028888068 2026-02-19T12:54:35.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:35.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-163208757251 2026-02-19T12:54:35.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:35.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:54:35.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-163208757251 2026-02-19T12:54:35.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:35.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757251 2026-02-19T12:54:35.745 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.8 seq 163208757251 2026-02-19T12:54:35.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 163208757251' 2026-02-19T12:54:35.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:54:35.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757251 -lt 163208757251 2026-02-19T12:54:35.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:35.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-180388626434 2026-02-19T12:54:35.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:35.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:54:35.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-180388626434 2026-02-19T12:54:35.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:35.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626434 2026-02-19T12:54:35.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 180388626434' 2026-02-19T12:54:35.917 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.9 seq 180388626434 2026-02-19T12:54:35.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:54:36.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626434 -lt 180388626434 2026-02-19T12:54:36.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:54:36.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:54:36.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:54:36.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:54:36.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:54:36.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:54:36.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:54:36.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:54:36.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:54:36.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:54:36.284 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:54:36.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:54:36.440 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:54:36.441 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:54:36.441 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:54:36.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:54:36.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:54:36.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:54:36.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:896: corrupt_and_repair_lrc: create_ec_pool ecpool false k=4 m=2 l=3 plugin=lrc 2026-02-19T12:54:36.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: create_ec_pool: local pool_name=ecpool 2026-02-19T12:54:36.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: create_ec_pool: shift 2026-02-19T12:54:36.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: create_ec_pool: local allow_overwrites=false 2026-02-19T12:54:36.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2525: create_ec_pool: shift 2026-02-19T12:54:36.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=4 m=2 l=3 plugin=lrc 2026-02-19T12:54:36.842 INFO:tasks.workunit.client.0.trial144.stderr:load: isa 2026-02-19T12:54:36.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-19T12:54:36.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-19T12:54:37.149 INFO:tasks.workunit.client.0.trial144.stderr:pool 'ecpool' created 2026-02-19T12:54:37.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:54:38.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: create_ec_pool: '[' false = true ']' 2026-02-19T12:54:38.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2535: create_ec_pool: wait_for_clean 2026-02-19T12:54:38.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:54:38.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:54:38.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:54:38.163 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:54:38.164 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:54:38.164 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:54:38.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:54:38.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:54:38.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:54:38.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:54:38.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:54:38.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:54:38.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:54:38.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:54:38.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:54:38.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:54:38.441 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:54:38.441 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:54:38.441 INFO:tasks.workunit.client.0.trial144.stderr:3 2026-02-19T12:54:38.441 INFO:tasks.workunit.client.0.trial144.stderr:4 2026-02-19T12:54:38.441 INFO:tasks.workunit.client.0.trial144.stderr:5 2026-02-19T12:54:38.441 INFO:tasks.workunit.client.0.trial144.stderr:6 2026-02-19T12:54:38.441 INFO:tasks.workunit.client.0.trial144.stderr:7 2026-02-19T12:54:38.442 INFO:tasks.workunit.client.0.trial144.stderr:8 2026-02-19T12:54:38.442 INFO:tasks.workunit.client.0.trial144.stderr:9' 2026-02-19T12:54:38.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:54:38.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:38.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:54:38.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-19T12:54:38.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-19T12:54:38.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491' 2026-02-19T12:54:38.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:38.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:54:38.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705674 2026-02-19T12:54:38.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705674 2026-02-19T12:54:38.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705674' 2026-02-19T12:54:38.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:38.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:54:38.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-19T12:54:38.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-19T12:54:38.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705674 2-55834574857' 2026-02-19T12:54:38.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:38.665 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:54:38.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411337 2026-02-19T12:54:38.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411337 2026-02-19T12:54:38.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705674 2-55834574857 3-77309411337' 2026-02-19T12:54:38.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:38.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:54:38.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280520 2026-02-19T12:54:38.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280520 2026-02-19T12:54:38.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705674 2-55834574857 3-77309411337 4-94489280520' 2026-02-19T12:54:38.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:38.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:54:38.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149703 2026-02-19T12:54:38.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149703 2026-02-19T12:54:38.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705674 2-55834574857 3-77309411337 4-94489280520 5-111669149703' 2026-02-19T12:54:38.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:38.882 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:54:38.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018886 2026-02-19T12:54:38.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018886 2026-02-19T12:54:38.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705674 2-55834574857 3-77309411337 4-94489280520 5-111669149703 6-128849018886' 2026-02-19T12:54:38.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:38.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:54:39.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888070 2026-02-19T12:54:39.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888070 2026-02-19T12:54:39.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705674 2-55834574857 3-77309411337 4-94489280520 5-111669149703 6-128849018886 7-146028888070' 2026-02-19T12:54:39.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:39.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:54:39.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757253 2026-02-19T12:54:39.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757253 2026-02-19T12:54:39.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705674 2-55834574857 3-77309411337 4-94489280520 5-111669149703 6-128849018886 7-146028888070 8-163208757253' 2026-02-19T12:54:39.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:39.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:54:39.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626436 2026-02-19T12:54:39.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626436 2026-02-19T12:54:39.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705674 2-55834574857 3-77309411337 4-94489280520 5-111669149703 6-128849018886 7-146028888070 8-163208757253 9-180388626436' 2026-02-19T12:54:39.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:39.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-19T12:54:39.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:39.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:54:39.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836491 2026-02-19T12:54:39.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:39.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836491 2026-02-19T12:54:39.177 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836491 2026-02-19T12:54:39.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-19T12:54:39.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:54:39.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T12:54:39.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:54:40.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:54:40.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:54:40.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-19T12:54:40.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:40.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705674 2026-02-19T12:54:40.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:40.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:54:40.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705674 2026-02-19T12:54:40.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:40.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705674 2026-02-19T12:54:40.509 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705674 2026-02-19T12:54:40.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-19T12:54:40.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:54:40.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705674 -lt 38654705674 2026-02-19T12:54:40.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:40.679 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574857 2026-02-19T12:54:40.679 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:40.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:54:40.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574857 2026-02-19T12:54:40.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:40.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574857 2026-02-19T12:54:40.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-19T12:54:40.683 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574857 2026-02-19T12:54:40.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:54:40.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574858 -lt 55834574857 2026-02-19T12:54:40.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:40.841 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411337 2026-02-19T12:54:40.841 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:40.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:54:40.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-77309411337 2026-02-19T12:54:40.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:40.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411337 2026-02-19T12:54:40.845 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.3 seq 77309411337 2026-02-19T12:54:40.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 77309411337' 2026-02-19T12:54:40.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:54:41.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411337 -lt 77309411337 2026-02-19T12:54:41.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:41.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280520 2026-02-19T12:54:41.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:41.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:54:41.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-94489280520 2026-02-19T12:54:41.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:41.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280520 2026-02-19T12:54:41.022 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.4 seq 94489280520 2026-02-19T12:54:41.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 94489280520' 2026-02-19T12:54:41.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:54:41.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280520 -lt 94489280520 2026-02-19T12:54:41.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:41.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-111669149703 2026-02-19T12:54:41.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:41.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:54:41.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-111669149703 2026-02-19T12:54:41.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:41.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149703 2026-02-19T12:54:41.186 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.5 seq 111669149703 2026-02-19T12:54:41.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 111669149703' 2026-02-19T12:54:41.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:54:41.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149703 -lt 111669149703 2026-02-19T12:54:41.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:41.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-128849018886 2026-02-19T12:54:41.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:41.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:54:41.351 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-128849018886 2026-02-19T12:54:41.351 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:41.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018886 2026-02-19T12:54:41.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 128849018886' 2026-02-19T12:54:41.353 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.6 seq 128849018886 2026-02-19T12:54:41.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:54:41.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018887 -lt 128849018886 2026-02-19T12:54:41.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:41.513 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-146028888070 2026-02-19T12:54:41.514 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:41.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:54:41.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-146028888070 2026-02-19T12:54:41.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:41.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888070 2026-02-19T12:54:41.517 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.7 seq 146028888070 2026-02-19T12:54:41.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 146028888070' 2026-02-19T12:54:41.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:54:41.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888070 -lt 146028888070 2026-02-19T12:54:41.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:41.690 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-163208757253 2026-02-19T12:54:41.690 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:41.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:54:41.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-163208757253 2026-02-19T12:54:41.693 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:41.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757253 2026-02-19T12:54:41.694 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.8 seq 163208757253 2026-02-19T12:54:41.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 163208757253' 2026-02-19T12:54:41.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:54:41.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757253 -lt 163208757253 2026-02-19T12:54:41.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:41.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-180388626436 2026-02-19T12:54:41.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:41.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:54:41.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-180388626436 2026-02-19T12:54:41.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:41.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626436 2026-02-19T12:54:41.873 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.9 seq 180388626436 2026-02-19T12:54:41.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 180388626436' 2026-02-19T12:54:41.873 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:54:42.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626436 -lt 180388626436 2026-02-19T12:54:42.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:54:42.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:54:42.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:54:42.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:54:42.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:54:42.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:54:42.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:54:42.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:54:42.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:54:42.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:54:42.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:54:42.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:54:42.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:54:42.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:54:42.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:54:42.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:54:42.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:54:42.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:54:42.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2536: create_ec_pool: return 0 2026-02-19T12:54:42.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:897: corrupt_and_repair_lrc: corrupt_and_repair_erasure_coded td/osd-scrub-repair ecpool 2026-02-19T12:54:42.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:308: corrupt_and_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-19T12:54:42.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:309: corrupt_and_repair_erasure_coded: local poolname=ecpool 2026-02-19T12:54:42.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:311: corrupt_and_repair_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-19T12:54:42.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T12:54:42.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T12:54:42.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-19T12:54:42.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T12:54:42.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T12:54:42.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T12:54:42.824 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T12:54:42.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T12:54:43.025 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T12:54:43.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T12:54:43.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T12:54:43.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-19T12:54:43.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:313: corrupt_and_repair_erasure_coded: get_primary ecpool SOMETHING 2026-02-19T12:54:43.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=ecpool 2026-02-19T12:54:43.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:54:43.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:54:43.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:54:43.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:313: corrupt_and_repair_erasure_coded: local primary=3 2026-02-19T12:54:43.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: get_osds ecpool SOMETHING 2026-02-19T12:54:43.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T12:54:43.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T12:54:43.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: sed -e s/3// 2026-02-19T12:54:43.227 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:54:43.228 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T12:54:43.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T12:54:43.393 INFO:tasks.workunit.client.0.trial144.stderr:5 2026-02-19T12:54:43.393 INFO:tasks.workunit.client.0.trial144.stderr:9 2026-02-19T12:54:43.393 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:54:43.393 INFO:tasks.workunit.client.0.trial144.stderr:6 2026-02-19T12:54:43.393 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:54:43.394 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:54:43.394 INFO:tasks.workunit.client.0.trial144.stderr:7' 2026-02-19T12:54:43.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 6 2 1 7 2026-02-19T12:54:43.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: osds=('5' '9' '0' '6' '2' '1' '7') 2026-02-19T12:54:43.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: local -a osds 2026-02-19T12:54:43.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:315: corrupt_and_repair_erasure_coded: local not_primary_first=5 2026-02-19T12:54:43.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:316: corrupt_and_repair_erasure_coded: local not_primary_second=9 2026-02-19T12:54:43.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:319: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 3 2026-02-19T12:54:43.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:286: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-19T12:54:43.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:287: corrupt_and_repair_one: local poolname=ecpool 2026-02-19T12:54:43.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:288: corrupt_and_repair_one: local osd=3 2026-02-19T12:54:43.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:293: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:54:43.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:54:43.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:54:43.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:54:43.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:54:43.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:54:43.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:54:43.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:54:43.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:54:43.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:54:43.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:54:43.398 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:54:43.398 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:54:43.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:54:43.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:54:43.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:54:43.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:54:43.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:54:43.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:54:43.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:54:43.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:54:43.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:54:43.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:54:43.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-19T12:54:43.894 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#2:eb822e21:::SOMETHING:head# 2026-02-19T12:54:44.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:54:44.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:54:44.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:54:44.007 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:54:44.008 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:44.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:44.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:54:44.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:54:44.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:54:44.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:54:44.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:54:44.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:54:44.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:54:44.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:54:44.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:54:44.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:54:44.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:54:44.009 INFO:tasks.workunit.client.0.trial144.stderr:start osd.3 2026-02-19T12:54:44.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:54:44.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:54:44.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:54:44.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:54:44.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:54:44.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:54:44.029 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:44.028+0000 7fcdc6d52a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:44.031 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:44.032+0000 7fcdc6d52a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:44.032 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:44.032+0000 7fcdc6d52a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:44.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:54:44.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:54:44.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:54:44.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:54:44.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:54:44.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:44.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:54:44.171 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:54:44.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:44.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:54:44.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:44.456 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:44.456+0000 7fcdc6d52a40 -1 Falling back to public interface 2026-02-19T12:54:45.140 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:45.140+0000 7fcdc6d52a40 -1 osd.3 57 log_to_monitors true 2026-02-19T12:54:45.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:45.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:45.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:54:45.331 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:54:45.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:45.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:54:45.489 INFO:tasks.workunit.client.0.trial144.stderr:osd.3 up in weight 1 up_from 61 up_thru 61 down_at 58 last_clean_interval [18,57) [v2:127.0.0.1:6826/192997776,v1:127.0.0.1:6827/192997776] [v2:127.0.0.1:6828/192997776,v1:127.0.0.1:6829/192997776] exists,up 55da20d9-a5e9-4e5b-8bcc-8d8f03ecbf6d 2026-02-19T12:54:45.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:54:45.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:54:45.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:54:45.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:54:45.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:54:45.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:54:45.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:54:45.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:54:45.491 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:54:45.491 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:54:45.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:54:45.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:54:45.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:54:45.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:54:45.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:54:45.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:54:45.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:54:45.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:54:45.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:54:45.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:54:45.766 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:54:45.767 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:54:45.767 INFO:tasks.workunit.client.0.trial144.stderr:3 2026-02-19T12:54:45.767 INFO:tasks.workunit.client.0.trial144.stderr:4 2026-02-19T12:54:45.767 INFO:tasks.workunit.client.0.trial144.stderr:5 2026-02-19T12:54:45.767 INFO:tasks.workunit.client.0.trial144.stderr:6 2026-02-19T12:54:45.767 INFO:tasks.workunit.client.0.trial144.stderr:7 2026-02-19T12:54:45.767 INFO:tasks.workunit.client.0.trial144.stderr:8 2026-02-19T12:54:45.767 INFO:tasks.workunit.client.0.trial144.stderr:9' 2026-02-19T12:54:45.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:54:45.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:45.768 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:54:45.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836493 2026-02-19T12:54:45.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836493 2026-02-19T12:54:45.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493' 2026-02-19T12:54:45.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:45.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:54:45.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705677 2026-02-19T12:54:45.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705677 2026-02-19T12:54:45.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-38654705677' 2026-02-19T12:54:45.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:45.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:54:45.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574860 2026-02-19T12:54:45.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574860 2026-02-19T12:54:45.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-38654705677 2-55834574860' 2026-02-19T12:54:45.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:45.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:54:46.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=261993005058 2026-02-19T12:54:46.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 261993005058 2026-02-19T12:54:46.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-38654705677 2-55834574860 3-261993005058' 2026-02-19T12:54:46.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:46.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:54:46.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280522 2026-02-19T12:54:46.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280522 2026-02-19T12:54:46.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-38654705677 2-55834574860 3-261993005058 4-94489280522' 2026-02-19T12:54:46.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:46.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:54:46.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149706 2026-02-19T12:54:46.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149706 2026-02-19T12:54:46.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-38654705677 2-55834574860 3-261993005058 4-94489280522 5-111669149706' 2026-02-19T12:54:46.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:46.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:54:46.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018889 2026-02-19T12:54:46.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018889 2026-02-19T12:54:46.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-38654705677 2-55834574860 3-261993005058 4-94489280522 5-111669149706 6-128849018889' 2026-02-19T12:54:46.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:46.274 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:54:46.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888072 2026-02-19T12:54:46.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888072 2026-02-19T12:54:46.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-38654705677 2-55834574860 3-261993005058 4-94489280522 5-111669149706 6-128849018889 7-146028888072' 2026-02-19T12:54:46.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:46.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:54:46.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757255 2026-02-19T12:54:46.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757255 2026-02-19T12:54:46.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-38654705677 2-55834574860 3-261993005058 4-94489280522 5-111669149706 6-128849018889 7-146028888072 8-163208757255' 2026-02-19T12:54:46.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:46.418 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:54:46.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626439 2026-02-19T12:54:46.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626439 2026-02-19T12:54:46.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-38654705677 2-55834574860 3-261993005058 4-94489280522 5-111669149706 6-128849018889 7-146028888072 8-163208757255 9-180388626439' 2026-02-19T12:54:46.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:46.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836493 2026-02-19T12:54:46.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:46.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:54:46.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836493 2026-02-19T12:54:46.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:46.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836493 2026-02-19T12:54:46.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-19T12:54:46.494 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836493 2026-02-19T12:54:46.494 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:54:46.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836493 -lt 21474836493 2026-02-19T12:54:46.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:46.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705677 2026-02-19T12:54:46.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:46.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:54:46.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705677 2026-02-19T12:54:46.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:46.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705677 2026-02-19T12:54:46.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705677' 2026-02-19T12:54:46.665 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 38654705677 2026-02-19T12:54:46.665 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:54:46.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705677 -lt 38654705677 2026-02-19T12:54:46.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:46.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574860 2026-02-19T12:54:46.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:46.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:54:46.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574860 2026-02-19T12:54:46.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:46.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574860 2026-02-19T12:54:46.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574860' 2026-02-19T12:54:46.835 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 55834574860 2026-02-19T12:54:46.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:54:46.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574860 -lt 55834574860 2026-02-19T12:54:46.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:46.993 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-261993005058 2026-02-19T12:54:46.993 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:46.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:54:46.995 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-261993005058 2026-02-19T12:54:46.995 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:46.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=261993005058 2026-02-19T12:54:46.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 261993005058' 2026-02-19T12:54:46.997 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 261993005058 2026-02-19T12:54:46.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:54:47.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 261993005058 -lt 261993005058 2026-02-19T12:54:47.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:47.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280522 2026-02-19T12:54:47.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:47.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:54:47.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-94489280522 2026-02-19T12:54:47.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:47.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280522 2026-02-19T12:54:47.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 94489280522' 2026-02-19T12:54:47.175 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.4 seq 94489280522 2026-02-19T12:54:47.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:54:47.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280521 -lt 94489280522 2026-02-19T12:54:47.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:54:48.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:54:48.339 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:54:48.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280523 -lt 94489280522 2026-02-19T12:54:48.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:48.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-111669149706 2026-02-19T12:54:48.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:48.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:54:48.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-111669149706 2026-02-19T12:54:48.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:48.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149706 2026-02-19T12:54:48.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 111669149706' 2026-02-19T12:54:48.502 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.5 seq 111669149706 2026-02-19T12:54:48.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:54:48.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149706 -lt 111669149706 2026-02-19T12:54:48.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:48.664 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-128849018889 2026-02-19T12:54:48.664 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:48.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:54:48.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-128849018889 2026-02-19T12:54:48.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:48.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018889 2026-02-19T12:54:48.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 128849018889' 2026-02-19T12:54:48.668 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.6 seq 128849018889 2026-02-19T12:54:48.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:54:48.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018889 -lt 128849018889 2026-02-19T12:54:48.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:48.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-146028888072 2026-02-19T12:54:48.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:48.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:54:48.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-146028888072 2026-02-19T12:54:48.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:48.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888072 2026-02-19T12:54:48.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 146028888072' 2026-02-19T12:54:48.837 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.7 seq 146028888072 2026-02-19T12:54:48.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:54:48.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888073 -lt 146028888072 2026-02-19T12:54:48.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:48.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-163208757255 2026-02-19T12:54:48.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:49.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:54:49.001 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-163208757255 2026-02-19T12:54:49.001 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:49.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757255 2026-02-19T12:54:49.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 163208757255' 2026-02-19T12:54:49.003 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.8 seq 163208757255 2026-02-19T12:54:49.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:54:49.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757256 -lt 163208757255 2026-02-19T12:54:49.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:49.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-180388626439 2026-02-19T12:54:49.163 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:49.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:54:49.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-180388626439 2026-02-19T12:54:49.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:49.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626439 2026-02-19T12:54:49.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 180388626439' 2026-02-19T12:54:49.167 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.9 seq 180388626439 2026-02-19T12:54:49.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:54:49.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626439 -lt 180388626439 2026-02-19T12:54:49.335 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:54:49.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:54:49.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:54:49.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:54:49.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:54:49.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:54:49.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:54:49.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:54:49.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:54:49.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:54:49.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:54:49.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:54:49.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:54:49.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:54:49.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:54:49.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:54:49.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:54:49.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:54:49.881 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-19T12:54:49.881 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:54:49.881 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:54:49.882 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:54:49.882 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:54:50.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: local pg=2.0 2026-02-19T12:54:50.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: corrupt_and_repair_one: repair 2.0 2026-02-19T12:54:50.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:54:50.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:54:50.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:54:50.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:54:50.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:54:50.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:54:50.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:54:37.148172+0000 2026-02-19T12:54:50.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:54:50.349 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:54:50.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:54:37.148172+0000 2026-02-19T12:54:50.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:54:50.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:54:37.148172+0000 2026-02-19T12:54:50.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:54:50.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:54:50.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:54:50.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:54:50.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:54:50.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:54:50.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:54:50.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:54:50.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:54:37.148172+0000 '>' 2026-02-19T12:54:37.148172+0000 2026-02-19T12:54:50.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:54:51.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:54:51.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:54:51.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:54:51.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:54:51.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:54:51.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:54:51.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:54:51.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:54:37.148172+0000 '>' 2026-02-19T12:54:37.148172+0000 2026-02-19T12:54:51.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:54:52.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:54:52.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:54:52.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:54:52.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:54:52.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:54:52.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:54:52.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:54:52.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:54:37.148172+0000 '>' 2026-02-19T12:54:37.148172+0000 2026-02-19T12:54:52.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:54:53.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:54:53.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:54:53.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:54:53.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:54:53.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:54:53.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:54:53.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:54:54.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:54:37.148172+0000 '>' 2026-02-19T12:54:37.148172+0000 2026-02-19T12:54:54.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:54:55.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:54:55.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:54:55.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:54:55.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:54:55.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:54:55.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:54:55.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:54:55.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:54:37.148172+0000 '>' 2026-02-19T12:54:37.148172+0000 2026-02-19T12:54:55.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:54:56.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:54:56.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:54:56.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:54:56.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:54:56.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:54:56.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:54:56.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:54:56.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:54:51.159031+0000 '>' 2026-02-19T12:54:37.148172+0000 2026-02-19T12:54:56.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:54:56.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:302: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:54:56.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:54:56.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:54:56.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-19T12:54:56.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:54:56.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:54:56.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:54:56.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:54:56.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-19T12:54:56.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:54:56.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-19T12:54:56.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:54:56.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:54:56.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:54:56.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:54:56.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:54:56.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:54:56.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:54:56.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:54:56.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:54:56.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-19T12:54:56.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:54:56.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:54:56.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-19T12:54:56.736 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:54:56.736 INFO:tasks.workunit.client.0.trial144.stdout:hinfo_key 2026-02-19T12:54:56.736 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:54:56.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-19T12:54:56.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:54:56.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:54:56.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-19T12:54:56.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:54:56.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:54:56.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:54:56.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:54:56.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:54:56.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:54:56.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-19T12:54:56.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-19T12:54:56.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:54:56.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:54:56.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:54:56.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:54:56.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:54:56.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:54:56.831 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:54:56.831 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:54:56.831 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:54:56.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:56.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:54:56.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:54:56.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:54:56.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:54:56.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:54:56.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:54:56.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:54:56.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:54:56.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:54:56.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:54:56.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-19T12:54:56.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-19T12:54:56.833 INFO:tasks.workunit.client.0.trial144.stderr:start osd.9 2026-02-19T12:54:56.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:54:56.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-19T12:54:56.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-19T12:54:56.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:54:56.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:54:56.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:54:56.852 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:56.851+0000 7f9fdfab5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:56.853 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:56.851+0000 7f9fdfab5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:56.854 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:56.855+0000 7f9fdfab5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:54:56.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-19T12:54:56.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:54:56.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-19T12:54:56.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:54:56.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:54:56.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:56.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:54:56.991 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:54:56.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:56.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:54:57.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:54:57.264 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:57.263+0000 7f9fdfab5a40 -1 Falling back to public interface 2026-02-19T12:54:57.884 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:54:57.883+0000 7f9fdfab5a40 -1 osd.9 64 log_to_monitors true 2026-02-19T12:54:58.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:54:58.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:54:58.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:54:58.151 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:54:58.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:54:58.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:54:58.310 INFO:tasks.workunit.client.0.trial144.stderr:osd.9 up in weight 1 up_from 67 up_thru 0 down_at 64 last_clean_interval [42,64) [v2:127.0.0.1:6874/1437075558,v1:127.0.0.1:6875/1437075558] [v2:127.0.0.1:6876/1437075558,v1:127.0.0.1:6877/1437075558] exists,up 19beb6b7-cd3d-4c41-8734-6c7b68c1a39d 2026-02-19T12:54:58.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:54:58.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:54:58.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:54:58.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:54:58.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:54:58.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:54:58.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:54:58.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:54:58.312 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:54:58.313 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:54:58.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:54:58.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:54:58.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:54:58.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:54:58.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:54:58.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:54:58.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:54:58.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:54:58.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:54:58.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:54:58.589 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:54:58.590 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:54:58.590 INFO:tasks.workunit.client.0.trial144.stderr:3 2026-02-19T12:54:58.590 INFO:tasks.workunit.client.0.trial144.stderr:4 2026-02-19T12:54:58.590 INFO:tasks.workunit.client.0.trial144.stderr:5 2026-02-19T12:54:58.590 INFO:tasks.workunit.client.0.trial144.stderr:6 2026-02-19T12:54:58.590 INFO:tasks.workunit.client.0.trial144.stderr:7 2026-02-19T12:54:58.590 INFO:tasks.workunit.client.0.trial144.stderr:8 2026-02-19T12:54:58.590 INFO:tasks.workunit.client.0.trial144.stderr:9' 2026-02-19T12:54:58.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:54:58.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:58.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:54:58.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836497 2026-02-19T12:54:58.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836497 2026-02-19T12:54:58.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497' 2026-02-19T12:54:58.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:58.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:54:58.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705680 2026-02-19T12:54:58.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705680 2026-02-19T12:54:58.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-38654705680' 2026-02-19T12:54:58.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:58.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:54:58.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574863 2026-02-19T12:54:58.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574863 2026-02-19T12:54:58.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863' 2026-02-19T12:54:58.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:58.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:54:58.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=261993005061 2026-02-19T12:54:58.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 261993005061 2026-02-19T12:54:58.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863 3-261993005061' 2026-02-19T12:54:58.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:58.881 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:54:58.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280526 2026-02-19T12:54:58.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280526 2026-02-19T12:54:58.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863 3-261993005061 4-94489280526' 2026-02-19T12:54:58.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:58.953 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:54:59.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149709 2026-02-19T12:54:59.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149709 2026-02-19T12:54:59.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863 3-261993005061 4-94489280526 5-111669149709' 2026-02-19T12:54:59.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:59.029 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:54:59.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018893 2026-02-19T12:54:59.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018893 2026-02-19T12:54:59.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863 3-261993005061 4-94489280526 5-111669149709 6-128849018893' 2026-02-19T12:54:59.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:59.101 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:54:59.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888076 2026-02-19T12:54:59.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888076 2026-02-19T12:54:59.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863 3-261993005061 4-94489280526 5-111669149709 6-128849018893 7-146028888076' 2026-02-19T12:54:59.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:59.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:54:59.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757259 2026-02-19T12:54:59.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757259 2026-02-19T12:54:59.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863 3-261993005061 4-94489280526 5-111669149709 6-128849018893 7-146028888076 8-163208757259' 2026-02-19T12:54:59.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:54:59.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:54:59.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808834 2026-02-19T12:54:59.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808834 2026-02-19T12:54:59.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863 3-261993005061 4-94489280526 5-111669149709 6-128849018893 7-146028888076 8-163208757259 9-287762808834' 2026-02-19T12:54:59.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:54:59.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836497 2026-02-19T12:54:59.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:54:59.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:54:59.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836497 2026-02-19T12:54:59.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:54:59.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836497 2026-02-19T12:54:59.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2026-02-19T12:54:59.319 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836497 2026-02-19T12:54:59.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:54:59.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836496 -lt 21474836497 2026-02-19T12:54:59.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:55:00.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:55:00.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:00.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836497 -lt 21474836497 2026-02-19T12:55:00.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:00.636 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705680 2026-02-19T12:55:00.636 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:00.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:55:00.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705680 2026-02-19T12:55:00.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:00.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705680 2026-02-19T12:55:00.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705680' 2026-02-19T12:55:00.640 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 38654705680 2026-02-19T12:55:00.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:55:00.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705680 -lt 38654705680 2026-02-19T12:55:00.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:00.807 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574863 2026-02-19T12:55:00.807 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:00.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:55:00.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574863 2026-02-19T12:55:00.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:00.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574863 2026-02-19T12:55:00.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574863' 2026-02-19T12:55:00.811 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 55834574863 2026-02-19T12:55:00.811 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:55:00.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574864 -lt 55834574863 2026-02-19T12:55:00.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:00.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-261993005061 2026-02-19T12:55:00.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:00.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:55:00.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-261993005061 2026-02-19T12:55:00.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:00.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=261993005061 2026-02-19T12:55:00.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 261993005061' 2026-02-19T12:55:00.975 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 261993005061 2026-02-19T12:55:00.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:55:01.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 261993005061 -lt 261993005061 2026-02-19T12:55:01.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:01.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280526 2026-02-19T12:55:01.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:01.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:55:01.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-94489280526 2026-02-19T12:55:01.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:01.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280526 2026-02-19T12:55:01.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 94489280526' 2026-02-19T12:55:01.136 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.4 seq 94489280526 2026-02-19T12:55:01.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:55:01.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280526 -lt 94489280526 2026-02-19T12:55:01.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:01.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-111669149709 2026-02-19T12:55:01.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:01.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:55:01.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-111669149709 2026-02-19T12:55:01.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:01.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149709 2026-02-19T12:55:01.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 111669149709' 2026-02-19T12:55:01.306 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.5 seq 111669149709 2026-02-19T12:55:01.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:55:01.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149709 -lt 111669149709 2026-02-19T12:55:01.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:01.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-128849018893 2026-02-19T12:55:01.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:01.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:55:01.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-128849018893 2026-02-19T12:55:01.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:01.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018893 2026-02-19T12:55:01.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 128849018893' 2026-02-19T12:55:01.469 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.6 seq 128849018893 2026-02-19T12:55:01.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:55:01.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018893 -lt 128849018893 2026-02-19T12:55:01.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:01.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-146028888076 2026-02-19T12:55:01.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:01.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:55:01.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-146028888076 2026-02-19T12:55:01.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:01.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888076 2026-02-19T12:55:01.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 146028888076' 2026-02-19T12:55:01.644 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.7 seq 146028888076 2026-02-19T12:55:01.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:55:01.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888076 -lt 146028888076 2026-02-19T12:55:01.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:01.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-163208757259 2026-02-19T12:55:01.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:01.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:55:01.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-163208757259 2026-02-19T12:55:01.816 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:01.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757259 2026-02-19T12:55:01.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 163208757259' 2026-02-19T12:55:01.817 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.8 seq 163208757259 2026-02-19T12:55:01.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:55:01.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757259 -lt 163208757259 2026-02-19T12:55:01.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:01.979 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-287762808834 2026-02-19T12:55:01.979 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:01.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:55:01.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-287762808834 2026-02-19T12:55:01.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:01.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=287762808834 2026-02-19T12:55:01.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 287762808834' 2026-02-19T12:55:01.983 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.9 seq 287762808834 2026-02-19T12:55:01.983 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:55:02.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 287762808834 -lt 287762808834 2026-02-19T12:55:02.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:55:02.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:02.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:02.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:55:02.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:02.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:02.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:02.341 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:02.341 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:02.341 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:02.341 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:02.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:55:02.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:02.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:02.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:02.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:55:02.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:55:02.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:55:02.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:55:02.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:304: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:321: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 5 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:286: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:287: corrupt_and_repair_one: local poolname=ecpool 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:288: corrupt_and_repair_one: local osd=5 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:293: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:55:02.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:55:02.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:55:03.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:55:03.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:55:03.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:55:03.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:55:03.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-19T12:55:03.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:55:03.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:55:03.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-19T12:55:03.210 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-19T12:55:03.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-19T12:55:03.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:55:03.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:55:03.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-19T12:55:03.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:55:03.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:55:03.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:55:03.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:55:03.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:55:03.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:55:03.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-19T12:55:03.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-19T12:55:03.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:55:03.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:55:03.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:55:03.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:55:03.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:55:03.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:55:03.327 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:55:03.327 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:03.327 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:03.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:03.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:03.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:55:03.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:55:03.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:55:03.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:55:03.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:55:03.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:55:03.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:55:03.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:55:03.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:55:03.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-19T12:55:03.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-19T12:55:03.330 INFO:tasks.workunit.client.0.trial144.stderr:start osd.5 2026-02-19T12:55:03.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:55:03.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-19T12:55:03.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-19T12:55:03.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:55:03.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:55:03.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:55:03.343 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:03.343+0000 7fd3011a6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:03.344 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:03.343+0000 7fd3011a6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:03.345 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:03.343+0000 7fd3011a6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:03.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-19T12:55:03.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:55:03.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T12:55:03.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:55:03.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:55:03.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:03.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:55:03.489 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:55:03.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:03.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:55:03.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:55:03.824 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:03.823+0000 7fd3011a6a40 -1 Falling back to public interface 2026-02-19T12:55:04.403 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:04.403+0000 7fd3011a6a40 -1 osd.5 69 log_to_monitors true 2026-02-19T12:55:04.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:55:04.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:04.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:55:04.649 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:55:04.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:04.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:55:04.806 INFO:tasks.workunit.client.0.trial144.stderr:osd.5 up in weight 1 up_from 72 up_thru 72 down_at 69 last_clean_interval [26,69) [v2:127.0.0.1:6842/3495331497,v1:127.0.0.1:6843/3495331497] [v2:127.0.0.1:6844/3495331497,v1:127.0.0.1:6845/3495331497] exists,up 2df300a0-d24a-42d1-b116-112bdb34ad84 2026-02-19T12:55:04.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:55:04.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:55:04.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:55:04.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:55:04.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:55:04.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:55:04.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:55:04.807 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:55:04.808 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:55:04.808 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:55:04.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:55:04.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:55:04.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:55:04.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:55:04.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:55:04.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:55:04.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:55:04.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:55:04.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:55:05.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:55:05.086 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:55:05.086 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:55:05.086 INFO:tasks.workunit.client.0.trial144.stderr:3 2026-02-19T12:55:05.086 INFO:tasks.workunit.client.0.trial144.stderr:4 2026-02-19T12:55:05.086 INFO:tasks.workunit.client.0.trial144.stderr:5 2026-02-19T12:55:05.086 INFO:tasks.workunit.client.0.trial144.stderr:6 2026-02-19T12:55:05.087 INFO:tasks.workunit.client.0.trial144.stderr:7 2026-02-19T12:55:05.087 INFO:tasks.workunit.client.0.trial144.stderr:8 2026-02-19T12:55:05.087 INFO:tasks.workunit.client.0.trial144.stderr:9' 2026-02-19T12:55:05.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:55:05.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:05.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:55:05.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836499 2026-02-19T12:55:05.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836499 2026-02-19T12:55:05.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499' 2026-02-19T12:55:05.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:05.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:55:05.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705682 2026-02-19T12:55:05.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705682 2026-02-19T12:55:05.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682' 2026-02-19T12:55:05.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:05.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:55:05.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574866 2026-02-19T12:55:05.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574866 2026-02-19T12:55:05.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866' 2026-02-19T12:55:05.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:05.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:55:05.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=261993005064 2026-02-19T12:55:05.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 261993005064 2026-02-19T12:55:05.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-261993005064' 2026-02-19T12:55:05.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:05.379 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:55:05.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280528 2026-02-19T12:55:05.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280528 2026-02-19T12:55:05.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-261993005064 4-94489280528' 2026-02-19T12:55:05.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:05.452 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:55:05.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645314 2026-02-19T12:55:05.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645314 2026-02-19T12:55:05.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-261993005064 4-94489280528 5-309237645314' 2026-02-19T12:55:05.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:05.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:55:05.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018895 2026-02-19T12:55:05.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018895 2026-02-19T12:55:05.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-261993005064 4-94489280528 5-309237645314 6-128849018895' 2026-02-19T12:55:05.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:05.597 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:55:05.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888078 2026-02-19T12:55:05.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888078 2026-02-19T12:55:05.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-261993005064 4-94489280528 5-309237645314 6-128849018895 7-146028888078' 2026-02-19T12:55:05.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:05.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:55:05.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757261 2026-02-19T12:55:05.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757261 2026-02-19T12:55:05.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-261993005064 4-94489280528 5-309237645314 6-128849018895 7-146028888078 8-163208757261' 2026-02-19T12:55:05.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:05.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:55:05.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808836 2026-02-19T12:55:05.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808836 2026-02-19T12:55:05.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-261993005064 4-94489280528 5-309237645314 6-128849018895 7-146028888078 8-163208757261 9-287762808836' 2026-02-19T12:55:05.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:05.816 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836499 2026-02-19T12:55:05.816 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:05.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:55:05.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836499 2026-02-19T12:55:05.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:05.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836499 2026-02-19T12:55:05.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2026-02-19T12:55:05.820 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836499 2026-02-19T12:55:05.820 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:05.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836498 -lt 21474836499 2026-02-19T12:55:05.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:55:06.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:55:06.985 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:07.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836499 -lt 21474836499 2026-02-19T12:55:07.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:07.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705682 2026-02-19T12:55:07.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:07.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:55:07.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705682 2026-02-19T12:55:07.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:07.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705682 2026-02-19T12:55:07.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705682' 2026-02-19T12:55:07.149 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 38654705682 2026-02-19T12:55:07.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:55:07.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705683 -lt 38654705682 2026-02-19T12:55:07.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:07.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574866 2026-02-19T12:55:07.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:07.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:55:07.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574866 2026-02-19T12:55:07.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:07.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574866 2026-02-19T12:55:07.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574866' 2026-02-19T12:55:07.321 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 55834574866 2026-02-19T12:55:07.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:55:07.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574866 -lt 55834574866 2026-02-19T12:55:07.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:07.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-261993005064 2026-02-19T12:55:07.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:07.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:55:07.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-261993005064 2026-02-19T12:55:07.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:07.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=261993005064 2026-02-19T12:55:07.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 261993005064' 2026-02-19T12:55:07.493 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 261993005064 2026-02-19T12:55:07.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:55:07.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 261993005064 -lt 261993005064 2026-02-19T12:55:07.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:07.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280528 2026-02-19T12:55:07.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:07.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:55:07.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-94489280528 2026-02-19T12:55:07.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:07.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280528 2026-02-19T12:55:07.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 94489280528' 2026-02-19T12:55:07.664 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.4 seq 94489280528 2026-02-19T12:55:07.664 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:55:07.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280528 -lt 94489280528 2026-02-19T12:55:07.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:07.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-309237645314 2026-02-19T12:55:07.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:07.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:55:07.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-309237645314 2026-02-19T12:55:07.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:07.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=309237645314 2026-02-19T12:55:07.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 309237645314' 2026-02-19T12:55:07.838 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.5 seq 309237645314 2026-02-19T12:55:07.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:55:08.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645314 -lt 309237645314 2026-02-19T12:55:08.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:08.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-128849018895 2026-02-19T12:55:08.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:08.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:55:08.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-128849018895 2026-02-19T12:55:08.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:08.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018895 2026-02-19T12:55:08.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 128849018895' 2026-02-19T12:55:08.011 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.6 seq 128849018895 2026-02-19T12:55:08.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:55:08.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018895 -lt 128849018895 2026-02-19T12:55:08.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:08.172 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-146028888078 2026-02-19T12:55:08.172 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:08.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:55:08.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-146028888078 2026-02-19T12:55:08.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:08.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888078 2026-02-19T12:55:08.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 146028888078' 2026-02-19T12:55:08.176 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.7 seq 146028888078 2026-02-19T12:55:08.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:55:08.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888079 -lt 146028888078 2026-02-19T12:55:08.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:08.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-163208757261 2026-02-19T12:55:08.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:08.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:55:08.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-163208757261 2026-02-19T12:55:08.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:08.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757261 2026-02-19T12:55:08.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 163208757261' 2026-02-19T12:55:08.351 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.8 seq 163208757261 2026-02-19T12:55:08.351 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:55:08.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757262 -lt 163208757261 2026-02-19T12:55:08.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:08.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-287762808836 2026-02-19T12:55:08.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:08.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:55:08.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-287762808836 2026-02-19T12:55:08.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:08.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=287762808836 2026-02-19T12:55:08.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 287762808836' 2026-02-19T12:55:08.526 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.9 seq 287762808836 2026-02-19T12:55:08.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:55:08.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 287762808837 -lt 287762808836 2026-02-19T12:55:08.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:55:08.693 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:08.693 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:08.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:55:08.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:08.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:08.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:08.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:08.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:08.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:08.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:09.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:55:09.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:09.045 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:09.045 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:09.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:55:09.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:55:09.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:55:09.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-19T12:55:09.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:55:09.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:55:09.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:55:09.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:55:09.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: local pg=2.0 2026-02-19T12:55:09.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: corrupt_and_repair_one: repair 2.0 2026-02-19T12:55:09.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:55:09.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:55:09.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:55:09.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:55:09.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:55:09.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:55:09.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:54:51.159031+0000 2026-02-19T12:55:09.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:55:09.712 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:55:09.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:54:51.159031+0000 2026-02-19T12:55:09.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:55:09.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:54:51.159031+0000 2026-02-19T12:55:09.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:55:09.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:55:09.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:55:09.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:55:09.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:55:09.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:55:09.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:55:09.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:55:09.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:54:51.159031+0000 '>' 2026-02-19T12:54:51.159031+0000 2026-02-19T12:55:09.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:55:10.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:55:10.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:55:10.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:55:10.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:55:10.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:55:10.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:55:10.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:55:11.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:54:51.159031+0000 '>' 2026-02-19T12:54:51.159031+0000 2026-02-19T12:55:11.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:55:12.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:55:12.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:55:12.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:55:12.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:55:12.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:55:12.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:55:12.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:55:12.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:55:09.791131+0000 '>' 2026-02-19T12:54:51.159031+0000 2026-02-19T12:55:12.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:55:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:302: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:55:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:55:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:55:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-19T12:55:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:55:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:55:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:55:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:55:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-19T12:55:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:55:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-19T12:55:12.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:55:12.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:55:12.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:55:12.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:55:12.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:55:12.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:55:12.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:55:12.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:55:12.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:55:12.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-19T12:55:12.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:55:12.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:55:12.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-19T12:55:12.585 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:55:12.585 INFO:tasks.workunit.client.0.trial144.stdout:hinfo_key 2026-02-19T12:55:12.585 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:55:12.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-19T12:55:12.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:55:12.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:55:12.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-19T12:55:12.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:55:12.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:55:12.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:55:12.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:55:12.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:55:12.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:55:12.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-19T12:55:12.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-19T12:55:12.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:55:12.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:55:12.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:55:12.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:55:12.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:55:12.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:55:12.675 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:55:12.675 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:12.676 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:12.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:12.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:12.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:55:12.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:55:12.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:55:12.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:55:12.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:55:12.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:55:12.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:55:12.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:55:12.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:55:12.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-19T12:55:12.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-19T12:55:12.677 INFO:tasks.workunit.client.0.trial144.stderr:start osd.9 2026-02-19T12:55:12.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:55:12.677 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-19T12:55:12.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-19T12:55:12.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:55:12.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:55:12.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:55:12.691 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:12.691+0000 7fc6704e9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:12.692 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:12.691+0000 7fc6704e9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:12.693 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:12.691+0000 7fc6704e9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:12.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-19T12:55:12.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:55:12.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-19T12:55:12.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:55:12.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:55:12.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:12.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:55:12.836 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:55:12.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:12.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:55:12.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:55:13.156 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:13.155+0000 7fc6704e9a40 -1 Falling back to public interface 2026-02-19T12:55:13.765 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:13.763+0000 7fc6704e9a40 -1 osd.9 74 log_to_monitors true 2026-02-19T12:55:13.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:55:13.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:13.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:55:13.995 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:55:13.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:13.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:55:14.160 INFO:tasks.workunit.client.0.trial144.stderr:osd.9 up in weight 1 up_from 78 up_thru 0 down_at 75 last_clean_interval [67,74) [v2:127.0.0.1:6874/3540514083,v1:127.0.0.1:6875/3540514083] [v2:127.0.0.1:6876/3540514083,v1:127.0.0.1:6877/3540514083] exists,up 19beb6b7-cd3d-4c41-8734-6c7b68c1a39d 2026-02-19T12:55:14.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:55:14.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:55:14.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:55:14.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:55:14.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:55:14.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:55:14.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:55:14.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:55:14.161 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:55:14.161 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:55:14.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:55:14.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:55:14.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:55:14.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:55:14.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:55:14.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:55:14.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:55:14.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:55:14.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:55:14.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:55:14.434 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:55:14.434 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:55:14.434 INFO:tasks.workunit.client.0.trial144.stderr:3 2026-02-19T12:55:14.434 INFO:tasks.workunit.client.0.trial144.stderr:4 2026-02-19T12:55:14.434 INFO:tasks.workunit.client.0.trial144.stderr:5 2026-02-19T12:55:14.434 INFO:tasks.workunit.client.0.trial144.stderr:6 2026-02-19T12:55:14.435 INFO:tasks.workunit.client.0.trial144.stderr:7 2026-02-19T12:55:14.435 INFO:tasks.workunit.client.0.trial144.stderr:8 2026-02-19T12:55:14.435 INFO:tasks.workunit.client.0.trial144.stderr:9' 2026-02-19T12:55:14.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:55:14.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:14.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:55:14.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836502 2026-02-19T12:55:14.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836502 2026-02-19T12:55:14.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502' 2026-02-19T12:55:14.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:14.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:55:14.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705685 2026-02-19T12:55:14.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705685 2026-02-19T12:55:14.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-38654705685' 2026-02-19T12:55:14.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:14.577 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:55:14.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574869 2026-02-19T12:55:14.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574869 2026-02-19T12:55:14.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-38654705685 2-55834574869' 2026-02-19T12:55:14.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:14.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:55:14.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=261993005066 2026-02-19T12:55:14.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 261993005066 2026-02-19T12:55:14.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-38654705685 2-55834574869 3-261993005066' 2026-02-19T12:55:14.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:14.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:55:14.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280531 2026-02-19T12:55:14.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280531 2026-02-19T12:55:14.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-38654705685 2-55834574869 3-261993005066 4-94489280531' 2026-02-19T12:55:14.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:14.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:55:14.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645317 2026-02-19T12:55:14.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645317 2026-02-19T12:55:14.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-38654705685 2-55834574869 3-261993005066 4-94489280531 5-309237645317' 2026-02-19T12:55:14.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:14.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:55:14.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018898 2026-02-19T12:55:14.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018898 2026-02-19T12:55:14.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-38654705685 2-55834574869 3-261993005066 4-94489280531 5-309237645317 6-128849018898' 2026-02-19T12:55:14.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:14.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:55:15.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888081 2026-02-19T12:55:15.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888081 2026-02-19T12:55:15.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-38654705685 2-55834574869 3-261993005066 4-94489280531 5-309237645317 6-128849018898 7-146028888081' 2026-02-19T12:55:15.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:15.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:55:15.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757264 2026-02-19T12:55:15.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757264 2026-02-19T12:55:15.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-38654705685 2-55834574869 3-261993005066 4-94489280531 5-309237645317 6-128849018898 7-146028888081 8-163208757264' 2026-02-19T12:55:15.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:15.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:55:15.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=335007449090 2026-02-19T12:55:15.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 335007449090 2026-02-19T12:55:15.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-38654705685 2-55834574869 3-261993005066 4-94489280531 5-309237645317 6-128849018898 7-146028888081 8-163208757264 9-335007449090' 2026-02-19T12:55:15.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:15.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836502 2026-02-19T12:55:15.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:15.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:55:15.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836502 2026-02-19T12:55:15.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:15.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836502 2026-02-19T12:55:15.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836502' 2026-02-19T12:55:15.169 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836502 2026-02-19T12:55:15.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:15.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836501 -lt 21474836502 2026-02-19T12:55:15.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:55:16.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:55:16.335 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:16.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836502 -lt 21474836502 2026-02-19T12:55:16.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:16.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705685 2026-02-19T12:55:16.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:16.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:55:16.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705685 2026-02-19T12:55:16.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:16.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705685 2026-02-19T12:55:16.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705685' 2026-02-19T12:55:16.502 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 38654705685 2026-02-19T12:55:16.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:55:16.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705686 -lt 38654705685 2026-02-19T12:55:16.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:16.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574869 2026-02-19T12:55:16.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:16.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:55:16.664 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574869 2026-02-19T12:55:16.664 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:16.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574869 2026-02-19T12:55:16.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574869' 2026-02-19T12:55:16.665 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 55834574869 2026-02-19T12:55:16.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:55:16.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574869 -lt 55834574869 2026-02-19T12:55:16.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:16.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-261993005066 2026-02-19T12:55:16.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:16.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:55:16.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-261993005066 2026-02-19T12:55:16.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:16.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=261993005066 2026-02-19T12:55:16.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 261993005066' 2026-02-19T12:55:16.839 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 261993005066 2026-02-19T12:55:16.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:55:16.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 261993005067 -lt 261993005066 2026-02-19T12:55:16.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:16.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280531 2026-02-19T12:55:16.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:17.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:55:17.001 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-94489280531 2026-02-19T12:55:17.001 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:17.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280531 2026-02-19T12:55:17.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 94489280531' 2026-02-19T12:55:17.003 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.4 seq 94489280531 2026-02-19T12:55:17.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:55:17.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280531 -lt 94489280531 2026-02-19T12:55:17.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:17.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-309237645317 2026-02-19T12:55:17.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:17.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:55:17.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-309237645317 2026-02-19T12:55:17.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:17.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=309237645317 2026-02-19T12:55:17.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 309237645317' 2026-02-19T12:55:17.163 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.5 seq 309237645317 2026-02-19T12:55:17.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:55:17.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645317 -lt 309237645317 2026-02-19T12:55:17.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:17.329 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-128849018898 2026-02-19T12:55:17.329 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:17.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:55:17.331 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-128849018898 2026-02-19T12:55:17.331 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:17.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018898 2026-02-19T12:55:17.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 128849018898' 2026-02-19T12:55:17.333 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.6 seq 128849018898 2026-02-19T12:55:17.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:55:17.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018898 -lt 128849018898 2026-02-19T12:55:17.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:17.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-146028888081 2026-02-19T12:55:17.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:17.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:55:17.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-146028888081 2026-02-19T12:55:17.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:17.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888081 2026-02-19T12:55:17.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 146028888081' 2026-02-19T12:55:17.497 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.7 seq 146028888081 2026-02-19T12:55:17.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:55:17.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888081 -lt 146028888081 2026-02-19T12:55:17.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:17.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-163208757264 2026-02-19T12:55:17.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:17.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:55:17.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-163208757264 2026-02-19T12:55:17.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:17.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757264 2026-02-19T12:55:17.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 163208757264' 2026-02-19T12:55:17.674 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.8 seq 163208757264 2026-02-19T12:55:17.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:55:17.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757264 -lt 163208757264 2026-02-19T12:55:17.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:17.842 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-335007449090 2026-02-19T12:55:17.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:17.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:55:17.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-335007449090 2026-02-19T12:55:17.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:17.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=335007449090 2026-02-19T12:55:17.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 335007449090' 2026-02-19T12:55:17.846 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.9 seq 335007449090 2026-02-19T12:55:17.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:55:18.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 335007449090 -lt 335007449090 2026-02-19T12:55:18.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:55:18.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:18.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:18.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:55:18.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:18.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:18.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:18.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:18.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:18.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:18.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:18.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:55:18.369 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:18.369 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:18.369 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:18.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:55:18.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:55:18.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:55:18.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:55:18.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:304: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:55:18.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:322: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 5 9 2026-02-19T12:55:18.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:245: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-19T12:55:18.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:246: corrupt_and_repair_two: local poolname=ecpool 2026-02-19T12:55:18.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:247: corrupt_and_repair_two: local first=5 2026-02-19T12:55:18.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_two: local second=9 2026-02-19T12:55:18.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_two: pids= 2026-02-19T12:55:18.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:55:18.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:55:18.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:55:18.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 184551"' 2026-02-19T12:55:18.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 184551' 2026-02-19T12:55:18.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 9 SOMETHING remove 2026-02-19T12:55:18.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:55:18.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:55:18.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 184553"' 2026-02-19T12:55:18.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 184553' 2026-02-19T12:55:18.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:55:18.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_two: wait_background pids 2026-02-19T12:55:18.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 184551 184553' 2026-02-19T12:55:18.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:55:18.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:55:18.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 184551 2026-02-19T12:55:18.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/184554: /' 2026-02-19T12:55:18.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 9 SOMETHING remove 2026-02-19T12:55:18.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/184556: /' 2026-02-19T12:55:19.238 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:55:19.239 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:55:19.239 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-19T12:55:19.239 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:55:19.239 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING remove 2026-02-19T12:55:19.239 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:55:19.239 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:55:19.239 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-19T12:55:19.239 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:55:19.240 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-19T12:55:19.240 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:55:19.240 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:55:19.240 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:55:19.240 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:55:19.240 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:55:19.240 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:55:19.240 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING remove 2026-02-19T12:55:19.240 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:55:19.240 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:55:19.241 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-19T12:55:19.241 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:55:19.241 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:55:19.241 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING remove 2026-02-19T12:55:19.241 INFO:tasks.workunit.client.0.trial144.stderr:184556: remove 2#2:eb822e21:::SOMETHING:head# 2026-02-19T12:55:19.241 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-19T12:55:19.241 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:55:19.241 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:55:19.241 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-19T12:55:19.241 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:55:19.241 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:55:19.241 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:55:19.242 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:55:19.242 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:55:19.243 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:55:19.243 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-19T12:55:19.243 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-19T12:55:19.243 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:55:19.243 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:55:19.243 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:55:19.243 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:55:19.243 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:55:19.243 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:55:19.243 INFO:tasks.workunit.client.0.trial144.stderr:184556: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:55:19.243 INFO:tasks.workunit.client.0.trial144.stderr:184556: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:19.243 INFO:tasks.workunit.client.0.trial144.stderr:184556: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:19.244 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:19.244 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:19.244 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:55:19.244 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:55:19.244 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:55:19.244 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:55:19.244 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:55:19.244 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:55:19.244 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:55:19.244 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:55:19.244 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:55:19.244 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-19T12:55:19.245 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-19T12:55:19.245 INFO:tasks.workunit.client.0.trial144.stderr:184556: start osd.9 2026-02-19T12:55:19.245 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:55:19.274 INFO:tasks.workunit.client.0.trial144.stderr:1184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:55:19.274 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:55:19.274 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-19T12:55:19.274 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:55:19.274 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:55:19.274 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:55:19.274 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:55:19.274 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-19T12:55:19.275 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:55:19.275 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-19T12:55:19.275 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:55:19.275 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:55:19.275 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:55:19.275 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:55:19.275 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:55:19.275 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:55:19.275 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:55:19.275 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:55:19.275 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:55:19.276 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-19T12:55:19.276 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:55:19.276 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:55:19.276 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-19T12:55:19.276 INFO:tasks.workunit.client.0.trial144.stderr:184554: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-19T12:55:19.276 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-19T12:55:19.276 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:55:19.276 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:55:19.276 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-19T12:55:19.276 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:55:19.277 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:55:19.277 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:55:19.277 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:55:19.277 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:55:19.278 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:55:19.278 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-19T12:55:19.278 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-19T12:55:19.278 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:55:19.278 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:55:19.278 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:55:19.278 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:55:19.278 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:55:19.278 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:55:19.278 INFO:tasks.workunit.client.0.trial144.stderr:184554: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:55:19.279 INFO:tasks.workunit.client.0.trial144.stderr:184554: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:19.279 INFO:tasks.workunit.client.0.trial144.stderr:184554: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:19.279 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:19.279 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:19.279 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:55:19.279 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:55:19.279 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:55:19.279 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:55:19.279 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:55:19.279 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:55:19.279 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:55:19.280 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:55:19.280 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:55:19.280 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-19T12:55:19.280 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-19T12:55:19.280 INFO:tasks.workunit.client.0.trial144.stderr:184554: start osd.5 2026-02-19T12:55:19.280 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:55:20.722 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-19T12:55:20.722 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-19T12:55:20.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:55:20.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:55:20.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:55:20.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: 2026-02-19T12:55:19.263+0000 7fb1071d5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:20.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: 2026-02-19T12:55:19.263+0000 7fb1071d5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:20.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: 2026-02-19T12:55:19.263+0000 7fb1071d5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:20.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-19T12:55:20.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:55:20.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-19T12:55:20.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:55:20.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:55:20.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:20.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:55:20.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: 0 2026-02-19T12:55:20.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:20.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:55:20.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:55:20.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: 2026-02-19T12:55:19.699+0000 7fb1071d5a40 -1 Falling back to public interface 2026-02-19T12:55:20.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: 2026-02-19T12:55:20.419+0000 7fb1071d5a40 -1 osd.9 79 log_to_monitors true 2026-02-19T12:55:20.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:55:20.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:20.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:55:20.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: 1 2026-02-19T12:55:20.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:20.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:55:20.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: osd.9 up in weight 1 up_from 83 up_thru 0 down_at 80 last_clean_interval [78,79) [v2:127.0.0.1:6842/1981325929,v1:127.0.0.1:6843/1981325929] [v2:127.0.0.1:6844/1981325929,v1:127.0.0.1:6845/1981325929] exists,up 19beb6b7-cd3d-4c41-8734-6c7b68c1a39d 2026-02-19T12:55:20.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:55:20.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:55:20.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:55:20.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:55:20.726 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:55:20.726 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:55:20.726 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:55:20.726 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:55:20.726 INFO:tasks.workunit.client.0.trial144.stderr:184556: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:55:20.761 INFO:tasks.workunit.client.0.trial144.stderr:184556: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: ge84554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-19T12:55:20.761 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-19T12:55:20.762 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:55:20.762 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:55:20.762 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:55:20.762 INFO:tasks.workunit.client.0.trial144.stderr:184554: 2026-02-19T12:55:19.291+0000 7fe447a70a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:20.762 INFO:tasks.workunit.client.0.trial144.stderr:184554: 2026-02-19T12:55:19.291+0000 7fe447a70a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:20.762 INFO:tasks.workunit.client.0.trial144.stderr:184554: 2026-02-19T12:55:19.291+0000 7fe447a70a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:20.762 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-19T12:55:20.762 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:55:20.762 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T12:55:20.762 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:55:20.763 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:55:20.763 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:20.763 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:55:20.763 INFO:tasks.workunit.client.0.trial144.stderr:184554: 0 2026-02-19T12:55:20.763 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:20.763 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:55:20.763 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:55:20.763 INFO:tasks.workunit.client.0.trial144.stderr:184554: 2026-02-19T12:55:19.711+0000 7fe447a70a40 -1 Falling back to public interface 2026-02-19T12:55:20.763 INFO:tasks.workunit.client.0.trial144.stderr:184554: 2026-02-19T12:55:20.419+0000 7fe447a70a40 -1 osd.5 79 log_to_monitors true 2026-02-19T12:55:20.763 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:55:20.763 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:20.764 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:55:20.764 INFO:tasks.workunit.client.0.trial144.stderr:184554: 1 2026-02-19T12:55:20.764 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:20.764 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:55:20.764 INFO:tasks.workunit.client.0.trial144.stderr:184554: osd.5 up in weight 1 up_from 83 up_thru 83 down_at 80 last_clean_interval [72,79) [v2:127.0.0.1:6874/2314178108,v1:127.0.0.1:6875/2314178108] [v2:127.0.0.1:6876/2314178108,v1:127.0.0.1:6877/2314178108] exists,up 2df300a0-d24a-42d1-b116-112bdb34ad84 2026-02-19T12:55:20.764 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:55:20.764 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:55:20.764 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:55:20.764 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:55:20.764 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:55:20.764 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:55:20.765 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:55:20.765 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:55:20.765 INFO:tasks.workunit.client.0.trial144.stderr:184554: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:55:21.279 INFO:tasks.workunit.client.0.trial144.stderr:184554: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: gt_timeout_delays: echo true 2026-02-19T12:55:21.280 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:55:21.280 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:55:21.280 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:55:21.280 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:55:21.280 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:55:21.280 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:55:21.280 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:55:21.280 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:55:21.281 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:55:21.281 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:55:21.281 INFO:tasks.workunit.client.0.trial144.stderr:184556: 1 2026-02-19T12:55:21.281 INFO:tasks.workunit.client.0.trial144.stderr:184556: 2 2026-02-19T12:55:21.281 INFO:tasks.workunit.client.0.trial144.stderr:184556: 3 2026-02-19T12:55:21.281 INFO:tasks.workunit.client.0.trial144.stderr:184556: 4 2026-02-19T12:55:21.281 INFO:tasks.workunit.client.0.trial144.stderr:184556: 5 2026-02-19T12:55:21.281 INFO:tasks.workunit.client.0.trial144.stderr:184556: 6 2026-02-19T12:55:21.281 INFO:tasks.workunit.client.0.trial144.stderr:184556: 7 2026-02-19T12:55:21.281 INFO:tasks.workunit.client.0.trial144.stderr:184556: 8 2026-02-19T12:55:21.281 INFO:tasks.workunit.client.0.trial144.stderr:184556: 9' 2026-02-19T12:55:21.282 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:55:21.282 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.282 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:55:21.282 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836504 2026-02-19T12:55:21.282 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836504 2026-02-19T12:55:21.282 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504' 2026-02-19T12:55:21.282 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.282 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:55:21.282 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705688 2026-02-19T12:55:21.282 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705688 2026-02-19T12:55:21.282 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-38654705688' 2026-02-19T12:55:21.283 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.283 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:55:21.283 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574871 2026-02-19T12:55:21.283 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574871 2026-02-19T12:55:21.283 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871' 2026-02-19T12:55:21.283 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.283 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:55:21.283 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=261993005069 2026-02-19T12:55:21.283 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 261993005069 2026-02-19T12:55:21.283 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871 3-261993005069' 2026-02-19T12:55:21.283 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.319 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: et_timeout_delays: echo true 2026-02-19T12:55:21.319 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:55:21.319 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:55:21.319 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:55:21.319 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:55:21.319 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:55:21.319 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:55:21.319 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:55:21.319 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:55:21.319 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:55:21.320 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:55:21.320 INFO:tasks.workunit.client.0.trial144.stderr:184554: 1 2026-02-19T12:55:21.320 INFO:tasks.workunit.client.0.trial144.stderr:184554: 2 2026-02-19T12:55:21.320 INFO:tasks.workunit.client.0.trial144.stderr:184554: 3 2026-02-19T12:55:21.320 INFO:tasks.workunit.client.0.trial144.stderr:184554: 4 2026-02-19T12:55:21.320 INFO:tasks.workunit.client.0.trial144.stderr:184554: 5 2026-02-19T12:55:21.320 INFO:tasks.workunit.client.0.trial144.stderr:184554: 6 2026-02-19T12:55:21.320 INFO:tasks.workunit.client.0.trial144.stderr:184554: 7 2026-02-19T12:55:21.320 INFO:tasks.workunit.client.0.trial144.stderr:184554: 8 2026-02-19T12:55:21.320 INFO:tasks.workunit.client.0.trial144.stderr:184554: 9' 2026-02-19T12:55:21.321 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:55:21.321 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.321 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:55:21.321 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836505 2026-02-19T12:55:21.321 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836505 2026-02-19T12:55:21.321 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505' 2026-02-19T12:55:21.321 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.321 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:55:21.321 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705689 2026-02-19T12:55:21.321 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705689 2026-02-19T12:55:21.321 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-38654705689' 2026-02-19T12:55:21.322 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.322 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:55:21.322 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574872 2026-02-19T12:55:21.322 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574872 2026-02-19T12:55:21.322 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-38654705689 2-55834574872' 2026-02-19T12:55:21.322 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.322 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:55:21.322 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=261993005070 2026-02-19T12:55:21.322 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 261993005070 2026-02-19T12:55:21.322 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-38654705689 2-55834574872 3-261993005070' 2026-02-19T12:55:21.322 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.722 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258:flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:55:21.722 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280533 2026-02-19T12:55:21.722 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280533 2026-02-19T12:55:21.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871 3-261993005069 4-94489280533' 2026-02-19T12:55:21.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:55:21.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285570 2026-02-19T12:55:21.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285570 2026-02-19T12:55:21.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871 3-261993005069 4-94489280533 5-356482285570' 2026-02-19T12:55:21.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:55:21.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018900 2026-02-19T12:55:21.723 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018900 2026-02-19T12:55:21.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871 3-261993005069 4-94489280533 5-356482285570 6-128849018900' 2026-02-19T12:55:21.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:55:21.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888083 2026-02-19T12:55:21.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888083 2026-02-19T12:55:21.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871 3-261993005069 4-94489280533 5-356482285570 6-128849018900 7-146028888083' 2026-02-19T12:55:21.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:55:21.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757267 2026-02-19T12:55:21.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757267 2026-02-19T12:55:21.724 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871 3-261993005069 4-94489280533 5-356482285570 6-128849018900 7-146028888083 8-163208757267' 2026-02-19T12:55:21.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:55:21.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285570 2026-02-19T12:55:21.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285570 2026-02-19T12:55:21.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871 3-261993005069 4-94489280533 5-356482285570 6-128849018900 7-146028888083 8-163208757267 9-356482285570' 2026-02-19T12:55:21.725 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:21.747 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/ceph flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:55:21.748 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280534 2026-02-19T12:55:21.748 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280534 2026-02-19T12:55:21.748 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-38654705689 2-55834574872 3-261993005070 4-94489280534' 2026-02-19T12:55:21.748 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.748 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:55:21.748 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285571 2026-02-19T12:55:21.748 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285571 2026-02-19T12:55:21.748 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-38654705689 2-55834574872 3-261993005070 4-94489280534 5-356482285571' 2026-02-19T12:55:21.749 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.749 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:55:21.749 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018901 2026-02-19T12:55:21.749 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018901 2026-02-19T12:55:21.749 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-38654705689 2-55834574872 3-261993005070 4-94489280534 5-356482285571 6-128849018901' 2026-02-19T12:55:21.749 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.749 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:55:21.749 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888084 2026-02-19T12:55:21.749 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888084 2026-02-19T12:55:21.749 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-38654705689 2-55834574872 3-261993005070 4-94489280534 5-356482285571 6-128849018901 7-146028888084' 2026-02-19T12:55:21.749 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.750 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:55:21.750 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757268 2026-02-19T12:55:21.750 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757268 2026-02-19T12:55:21.750 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-38654705689 2-55834574872 3-261993005070 4-94489280534 5-356482285571 6-128849018901 7-146028888084 8-163208757268' 2026-02-19T12:55:21.750 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:21.750 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:55:21.750 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285571 2026-02-19T12:55:21.750 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285571 2026-02-19T12:55:21.750 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-38654705689 2-55834574872 3-261993005070 4-94489280534 5-356482285571 6-128849018901 7-146028888084 8-163208757268 9-356482285571' 2026-02-19T12:55:21.750 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:23.381 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/ceptest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836504 2026-02-19T12:55:23.381 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:23.381 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:55:23.382 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836504 2026-02-19T12:55:23.382 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:23.382 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836504 2026-02-19T12:55:23.382 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836504' 2026-02-19T12:55:23.382 INFO:tasks.workunit.client.0.trial144.stderr:184556: waiting osd.0 seq 21474836504 2026-02-19T12:55:23.382 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:23.382 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836503 -lt 21474836504 2026-02-19T12:55:23.382 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:55:23.382 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:55:23.382 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:23.382 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836506 -lt 21474836504 2026-02-19T12:55:23.383 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:23.383 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705688 2026-02-19T12:55:23.383 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:23.383 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:55:23.383 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705688 2026-02-19T12:55:23.383 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:23.383 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705688 2026-02-19T12:55:23.383 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705688' 2026-02-19T12:55:23.383 INFO:tasks.workunit.client.0.trial144.stderr:184556: waiting osd.1 seq 38654705688 2026-02-19T12:55:23.383 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:55:23.383 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705689 -lt 38654705688 2026-02-19T12:55:23.383 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:23.383 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574871 2026-02-19T12:55:23.384 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:23.384 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:55:23.384 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574871 2026-02-19T12:55:23.384 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:23.384 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574871 2026-02-19T12:55:23.384 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574871' 2026-02-19T12:55:23.384 INFO:tasks.workunit.client.0.trial144.stderr:184556: waiting osd.2 seq 55834574871 2026-02-19T12:55:23.384 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:55:23.384 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574872 -lt 55834574871 2026-02-19T12:55:23.387 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_statshtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836505 2026-02-19T12:55:23.387 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:23.387 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:55:23.387 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836505 2026-02-19T12:55:23.387 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:23.387 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836505 2026-02-19T12:55:23.387 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836505' 2026-02-19T12:55:23.388 INFO:tasks.workunit.client.0.trial144.stderr:184554: waiting osd.0 seq 21474836505 2026-02-19T12:55:23.388 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:23.388 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836503 -lt 21474836505 2026-02-19T12:55:23.388 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:55:23.388 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:55:23.388 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:23.388 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836506 -lt 21474836505 2026-02-19T12:55:23.388 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:23.388 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705689 2026-02-19T12:55:23.388 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:23.388 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:55:23.388 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705689 2026-02-19T12:55:23.388 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:23.389 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705689 2026-02-19T12:55:23.389 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705689' 2026-02-19T12:55:23.389 INFO:tasks.workunit.client.0.trial144.stderr:184554: waiting osd.1 seq 38654705689 2026-02-19T12:55:23.389 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:55:23.389 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705689 -lt 38654705689 2026-02-19T12:55:23.389 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:23.389 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574872 2026-02-19T12:55:23.389 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:23.389 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:55:23.389 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574872 2026-02-19T12:55:23.389 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:23.389 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574872 2026-02-19T12:55:23.389 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574872' 2026-02-19T12:55:23.390 INFO:tasks.workunit.client.0.trial144.stderr:184554: waiting osd.2 seq 55834574872 2026-02-19T12:55:23.390 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:55:23.390 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574872 -lt 55834574872 2026-02-19T12:55:23.890 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stat: for s in $seqs 2026-02-19T12:55:23.891 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-261993005069 2026-02-19T12:55:23.891 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:23.891 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:55:23.891 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-261993005069 2026-02-19T12:55:23.891 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:23.891 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=261993005069 2026-02-19T12:55:23.891 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 261993005069' 2026-02-19T12:55:23.891 INFO:tasks.workunit.client.0.trial144.stderr:184556: waiting osd.3 seq 261993005069 2026-02-19T12:55:23.891 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:55:23.891 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 261993005070 -lt 261993005069 2026-02-19T12:55:23.892 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:23.892 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280533 2026-02-19T12:55:23.892 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:23.892 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:55:23.892 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-94489280533 2026-02-19T12:55:23.892 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:23.892 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280533 2026-02-19T12:55:23.892 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 94489280533' 2026-02-19T12:55:23.892 INFO:tasks.workunit.client.0.trial144.stderr:184556: waiting osd.4 seq 94489280533 2026-02-19T12:55:23.892 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:55:23.892 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280535 -lt 94489280533 2026-02-19T12:55:23.892 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:23.893 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-356482285570 2026-02-19T12:55:23.893 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:23.893 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:55:23.893 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-356482285570 2026-02-19T12:55:23.893 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:23.893 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285570 2026-02-19T12:55:23.893 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 356482285570' 2026-02-19T12:55:23.893 INFO:tasks.workunit.client.0.trial144.stderr:184556: waiting osd.5 seq 356482285570 2026-02-19T12:55:23.893 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:55:23.893 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285571 -lt 356482285570 2026-02-19T12:55:23.893 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:23.893 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-128849018900 2026-02-19T12:55:23.893 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:23.894 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:55:23.901 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-s: for s in $seqs 2026-02-19T12:55:23.901 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-261993005070 2026-02-19T12:55:23.901 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:23.901 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:55:23.901 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-261993005070 2026-02-19T12:55:23.901 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:23.901 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=261993005070 2026-02-19T12:55:23.901 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 261993005070' 2026-02-19T12:55:23.901 INFO:tasks.workunit.client.0.trial144.stderr:184554: waiting osd.3 seq 261993005070 2026-02-19T12:55:23.902 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:55:23.902 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 261993005070 -lt 261993005070 2026-02-19T12:55:23.902 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:23.902 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280534 2026-02-19T12:55:23.902 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:23.902 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:55:23.902 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-94489280534 2026-02-19T12:55:23.902 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:23.902 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280534 2026-02-19T12:55:23.902 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 94489280534' 2026-02-19T12:55:23.902 INFO:tasks.workunit.client.0.trial144.stderr:184554: waiting osd.4 seq 94489280534 2026-02-19T12:55:23.902 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:55:23.902 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280535 -lt 94489280534 2026-02-19T12:55:23.903 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:23.903 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-356482285571 2026-02-19T12:55:23.903 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:23.903 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:55:23.903 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-356482285571 2026-02-19T12:55:23.903 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:23.903 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285571 2026-02-19T12:55:23.903 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 356482285571' 2026-02-19T12:55:23.903 INFO:tasks.workunit.client.0.trial144.stderr:184554: waiting osd.5 seq 356482285571 2026-02-19T12:55:23.903 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:55:23.903 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285571 -lt 356482285571 2026-02-19T12:55:23.903 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:23.904 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-128849018901 2026-02-19T12:55:23.904 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:23.904 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:55:24.388 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/cephhelpers.sh:2268: flush_pg_stats: echo 6-128849018900 2026-02-19T12:55:24.388 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:24.388 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018900 2026-02-19T12:55:24.388 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 128849018900' 2026-02-19T12:55:24.388 INFO:tasks.workunit.client.0.trial144.stderr:184556: waiting osd.6 seq 128849018900 2026-02-19T12:55:24.389 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:55:24.389 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018901 -lt 128849018900 2026-02-19T12:55:24.389 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:24.389 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-146028888083 2026-02-19T12:55:24.389 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:24.389 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:55:24.389 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-146028888083 2026-02-19T12:55:24.389 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:24.389 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888083 2026-02-19T12:55:24.389 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 146028888083' 2026-02-19T12:55:24.389 INFO:tasks.workunit.client.0.trial144.stderr:184556: waiting osd.7 seq 146028888083 2026-02-19T12:55:24.389 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:55:24.390 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888085 -lt 146028888083 2026-02-19T12:55:24.390 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:24.390 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-163208757267 2026-02-19T12:55:24.390 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:24.390 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:55:24.390 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-163208757267 2026-02-19T12:55:24.390 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:24.390 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757267 2026-02-19T12:55:24.390 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 163208757267' 2026-02-19T12:55:24.390 INFO:tasks.workunit.client.0.trial144.stderr:184556: waiting osd.8 seq 163208757267 2026-02-19T12:55:24.390 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:55:24.390 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757268 -lt 163208757267 2026-02-19T12:55:24.391 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:24.391 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-356482285570 2026-02-19T12:55:24.391 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:24.391 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:55:24.391 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-356482285570 2026-02-19T12:55:24.391 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:24.391 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285570 2026-02-19T12:55:24.395 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 35648228-helpers.sh:2268: flush_pg_stats: echo 6-128849018901 2026-02-19T12:55:24.395 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:24.395 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018901 2026-02-19T12:55:24.395 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 128849018901' 2026-02-19T12:55:24.395 INFO:tasks.workunit.client.0.trial144.stderr:184554: waiting osd.6 seq 128849018901 2026-02-19T12:55:24.395 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:55:24.395 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018901 -lt 128849018901 2026-02-19T12:55:24.395 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:24.395 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-146028888084 2026-02-19T12:55:24.396 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:24.396 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:55:24.396 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-146028888084 2026-02-19T12:55:24.396 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:24.396 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888084 2026-02-19T12:55:24.396 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 146028888084' 2026-02-19T12:55:24.396 INFO:tasks.workunit.client.0.trial144.stderr:184554: waiting osd.7 seq 146028888084 2026-02-19T12:55:24.396 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:55:24.396 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888085 -lt 146028888084 2026-02-19T12:55:24.396 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:24.396 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-163208757268 2026-02-19T12:55:24.396 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:24.396 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:55:24.397 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-163208757268 2026-02-19T12:55:24.397 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:24.397 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757268 2026-02-19T12:55:24.397 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 163208757268' 2026-02-19T12:55:24.397 INFO:tasks.workunit.client.0.trial144.stderr:184554: waiting osd.8 seq 163208757268 2026-02-19T12:55:24.397 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:55:24.397 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757268 -lt 163208757268 2026-02-19T12:55:24.397 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:24.397 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-356482285571 2026-02-19T12:55:24.397 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:24.397 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:55:24.397 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-356482285571 2026-02-19T12:55:24.398 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:24.398 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285571 2026-02-19T12:55:25.363 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 35648225570' 2026-02-19T12:55:25.363 INFO:tasks.workunit.client.0.trial144.stderr:184556: waiting osd.9 seq 356482285570 2026-02-19T12:55:25.363 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:55:25.363 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285571 -lt 356482285570 2026-02-19T12:55:25.363 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:55:25.363 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:25.363 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:25.363 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:55:25.363 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:25.363 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:25.363 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:25.363 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:25.364 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:25.365 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:25.365 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:25.382 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.085571' 2026-02-19T12:55:25.382 INFO:tasks.workunit.client.0.trial144.stderr:184554: waiting osd.9 seq 356482285571 2026-02-19T12:55:25.382 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:55:25.382 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285571 -lt 356482285571 2026-02-19T12:55:25.382 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:55:25.382 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:25.382 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:25.382 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:55:25.382 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:25.382 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:25.383 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:25.383 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:25.383 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:25.383 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:25.383 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:25.383 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:55:25.383 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:25.383 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:25.383 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:25.383 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:55:25.383 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-19T12:55:25.383 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T12:55:25.383 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-19T12:55:25.384 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:55:25.384 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T12:55:25.384 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:55:25.384 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:25.384 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:25.384 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:25.384 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:25.384 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:25.384 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:25.384 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:26.305 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client./qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:55:26.305 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:26.305 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:26.305 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:26.305 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:55:26.305 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T12:55:26.305 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:55:26.306 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:55:26.306 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:55:26.306 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:55:26.306 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:55:26.306 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:55:26.306 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:55:26.306 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:55:26.306 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:55:26.306 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T12:55:26.306 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:55:26.306 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T12:55:26.306 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:55:26.307 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:26.307 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:26.307 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:26.307 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:26.307 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:26.307 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:26.307 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:26.307 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:55:26.307 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:26.307 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:26.307 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:26.307 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:55:26.330 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:55:26.330 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:26.330 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:26.330 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:26.330 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:55:26.331 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T12:55:26.331 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:55:26.331 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:55:26.331 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:55:26.331 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:55:26.331 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:55:26.331 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:55:26.331 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:55:26.331 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:55:26.331 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:55:26.331 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T12:55:26.331 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:55:26.332 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T12:55:26.332 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:55:26.332 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:26.332 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:26.332 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:26.332 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:26.332 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:26.332 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:26.332 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:26.332 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:55:26.332 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:26.332 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:26.332 INFO:tasks.workunit.client.0.trial144.stderr:184554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:26.333 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:55:26.333 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:55:26.333 INFO:tasks.workunit.client.0.trial144.stderr:184554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:55:26.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:55:26.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:55:26.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 184553 2026-02-19T12:55:27.253 INFO:tasks.workunit.client.0.trial144.stderr:/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T12:55:27.253 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:55:27.253 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:55:27.253 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:55:27.253 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:55:27.254 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:55:27.254 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:55:27.254 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:55:27.254 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:55:27.254 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:55:27.254 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-19T12:55:27.254 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:55:27.254 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-19T12:55:27.254 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:55:27.254 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:27.254 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:27.254 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:27.254 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:27.255 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:27.255 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:27.255 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:27.255 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:55:27.255 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:27.255 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:27.255 INFO:tasks.workunit.client.0.trial144.stderr:184556: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:27.255 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:55:27.255 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:55:27.255 INFO:tasks.workunit.client.0.trial144.stderr:184556: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:55:27.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:55:27.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:55:27.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:55:27.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:55:27.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:257: corrupt_and_repair_two: return_code=0 2026-02-19T12:55:27.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:258: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:55:27.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-19T12:55:27.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:55:27.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:55:27.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:55:27.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:55:27.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: local pg=2.0 2026-02-19T12:55:27.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:264: corrupt_and_repair_two: repair 2.0 2026-02-19T12:55:27.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:55:27.417 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:55:27.417 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:55:27.417 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:55:27.417 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:55:27.417 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:55:27.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:55:09.791131+0000 2026-02-19T12:55:27.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:55:27.721 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:55:27.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:55:09.791131+0000 2026-02-19T12:55:27.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:55:27.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:55:09.791131+0000 2026-02-19T12:55:27.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:55:27.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:55:27.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:55:27.732 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:55:27.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:55:27.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:55:27.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:55:27.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:55:27.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:55:09.791131+0000 '>' 2026-02-19T12:55:09.791131+0000 2026-02-19T12:55:27.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:55:28.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:55:28.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:55:28.893 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:55:28.893 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:55:28.893 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:55:28.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:55:28.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:55:29.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:55:09.791131+0000 '>' 2026-02-19T12:55:09.791131+0000 2026-02-19T12:55:29.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:55:30.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:55:30.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:55:30.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:55:30.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:55:30.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:55:30.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:55:30.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:55:30.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:55:09.791131+0000 '>' 2026-02-19T12:55:09.791131+0000 2026-02-19T12:55:30.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:55:31.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:55:31.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:55:31.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:55:31.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:55:31.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:55:31.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:55:31.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:55:31.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:55:09.791131+0000 '>' 2026-02-19T12:55:09.791131+0000 2026-02-19T12:55:31.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:55:32.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:55:32.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:55:32.371 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:55:32.371 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:55:32.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:55:32.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:55:32.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:55:32.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:55:28.659912+0000 '>' 2026-02-19T12:55:09.791131+0000 2026-02-19T12:55:32.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:55:32.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:268: corrupt_and_repair_two: pids= 2026-02-19T12:55:32.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:269: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:55:32.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:55:32.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:55:32.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 188280"' 2026-02-19T12:55:32.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 188280' 2026-02-19T12:55:32.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:270: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:55:32.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:55:32.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:55:32.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 188281"' 2026-02-19T12:55:32.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 188281' 2026-02-19T12:55:32.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:271: corrupt_and_repair_two: wait_background pids 2026-02-19T12:55:32.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 188280 188281' 2026-02-19T12:55:32.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:55:32.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:55:32.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 188280 2026-02-19T12:55:32.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:55:32.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/188283: /' 2026-02-19T12:55:32.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:55:32.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/188285: /' 2026-02-19T12:55:33.018 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:55:33.018 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:55:33.018 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-19T12:55:33.018 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:55:33.018 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:55:33.018 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:55:33.018 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:55:33.018 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-19T12:55:33.018 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:55:33.018 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-19T12:55:33.018 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:55:33.018 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:55:33.018 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:55:33.018 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:55:33.019 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:55:33.019 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:55:33.019 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:55:33.019 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:55:33.019 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:55:33.019 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-19T12:55:33.019 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:55:33.019 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:55:33.019 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING list-attrs 2026-02-19T12:55:33.019 INFO:tasks.workunit.client.0.trial144.stderr:188283: _ 2026-02-19T12:55:33.019 INFO:tasks.workunit.client.0.trial144.stderr:188283: hinfo_key 2026-02-19T12:55:33.019 INFO:tasks.workunit.client.0.trial144.stderr:188283: snapset 2026-02-19T12:55:33.019 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-19T12:55:33.020 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:55:33.020 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:55:33.020 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-19T12:55:33.020 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:55:33.020 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:55:33.020 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:55:33.020 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:55:33.021 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:55:33.021 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:55:33.021 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-19T12:55:33.021 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-19T12:55:33.021 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:55:33.021 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:55:33.021 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:55:33.022 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:55:33.022 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:55:33.022 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:55:33.022 INFO:tasks.workunit.client.0.trial144.stderr:188283: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:55:33.022 INFO:tasks.workunit.client.0.trial144.stderr:188283: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:33.022 INFO:tasks.workunit.client.0.trial144.stderr:188283: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:33.022 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:33.022 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:33.022 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:55:33.022 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:55:33.022 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:55:33.022 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:55:33.022 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:55:33.023 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:55:33.023 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:55:33.023 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:55:33.023 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:55:33.023 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-19T12:55:33.023 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-19T12:55:33.023 INFO:tasks.workunit.client.0.trial144.stderr:188283: start osd.5 2026-02-19T12:55:33.034 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_reco188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:55:33.034 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:55:33.034 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-19T12:55:33.034 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:55:33.034 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:55:33.034 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:55:33.034 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:55:33.035 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-19T12:55:33.035 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:55:33.035 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-19T12:55:33.035 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:55:33.035 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:55:33.035 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:55:33.035 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:55:33.035 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:55:33.035 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:55:33.035 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:55:33.035 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:55:33.035 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:55:33.036 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-19T12:55:33.036 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:55:33.036 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:55:33.036 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-19T12:55:33.036 INFO:tasks.workunit.client.0.trial144.stderr:188285: _ 2026-02-19T12:55:33.036 INFO:tasks.workunit.client.0.trial144.stderr:188285: hinfo_key 2026-02-19T12:55:33.036 INFO:tasks.workunit.client.0.trial144.stderr:188285: snapset 2026-02-19T12:55:33.036 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-19T12:55:33.036 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:55:33.036 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:55:33.036 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-19T12:55:33.036 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:55:33.036 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:55:33.037 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:55:33.037 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:55:33.038 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:55:33.038 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:55:33.038 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-19T12:55:33.038 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-19T12:55:33.038 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:55:33.038 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:55:33.039 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:55:33.039 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:55:33.039 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:55:33.039 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:55:33.039 INFO:tasks.workunit.client.0.trial144.stderr:188285: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:55:33.039 INFO:tasks.workunit.client.0.trial144.stderr:188285: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:33.039 INFO:tasks.workunit.client.0.trial144.stderr:188285: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:33.039 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:33.039 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:33.039 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:55:33.039 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:55:33.039 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:55:33.039 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:55:33.040 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:55:33.040 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:55:33.040 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:55:33.040 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:55:33.040 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:55:33.040 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-19T12:55:33.040 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-19T12:55:33.040 INFO:tasks.workunit.client.0.trial144.stderr:188285: start osd.9 2026-02-19T12:55:34.512 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:55:34.512 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-19T12:55:34.512 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-19T12:55:34.512 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:55:34.513 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:55:34.513 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:55:34.513 INFO:tasks.workunit.client.0.trial144.stderr:188283: 2026-02-19T12:55:33.039+0000 7f44f3ab1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:34.513 INFO:tasks.workunit.client.0.trial144.stderr:188283: 2026-02-19T12:55:33.039+0000 7f44f3ab1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:34.513 INFO:tasks.workunit.client.0.trial144.stderr:188283: 2026-02-19T12:55:33.039+0000 7f44f3ab1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:34.513 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-19T12:55:34.513 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:55:34.513 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T12:55:34.513 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:55:34.513 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:55:34.513 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:34.513 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:55:34.513 INFO:tasks.workunit.client.0.trial144.stderr:188283: 0 2026-02-19T12:55:34.514 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:34.514 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:55:34.514 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:55:34.514 INFO:tasks.workunit.client.0.trial144.stderr:188283: 2026-02-19T12:55:33.543+0000 7f44f3ab1a40 -1 Falling back to public interface 2026-02-19T12:55:34.514 INFO:tasks.workunit.client.0.trial144.stderr:188283: 2026-02-19T12:55:34.239+0000 7f44f3ab1a40 -1 osd.5 84 log_to_monitors true 2026-02-19T12:55:34.514 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:55:34.514 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:34.514 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:55:34.514 INFO:tasks.workunit.client.0.trial144.stderr:188283: 1 2026-02-19T12:55:34.514 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:34.514 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:55:34.514 INFO:tasks.workunit.client.0.trial144.stderr:188283: osd.5 up in weight 1 up_from 88 up_thru 88 down_at 85 last_clean_interval [83,84) [v2:127.0.0.1:6874/1555613964,v1:127.0.0.1:6875/1555613964] [v2:127.0.0.1:6876/1555613964,v1:127.0.0.1:6877/1555613964] exists,up 2df300a0-d24a-42d1-b116-112bdb34ad84 2026-02-19T12:55:34.515 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:55:34.515 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:55:34.515 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:55:34.515 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:55:34.515 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:55:34.515 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:55:34.515 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:55:34.515 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:55:34.515 INFO:tasks.workunit.client.0.trial144.stderr:188283: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:55:34.516 INFO:tasks.workunit.client.0.trial144.stderr:188283: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.very_ops 2026-02-19T12:55:34.516 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-19T12:55:34.516 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-19T12:55:34.516 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:55:34.516 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:55:34.516 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:55:34.516 INFO:tasks.workunit.client.0.trial144.stderr:188285: 2026-02-19T12:55:33.055+0000 7fe327934a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:34.516 INFO:tasks.workunit.client.0.trial144.stderr:188285: 2026-02-19T12:55:33.055+0000 7fe327934a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:34.516 INFO:tasks.workunit.client.0.trial144.stderr:188285: 2026-02-19T12:55:33.059+0000 7fe327934a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:34.517 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-19T12:55:34.517 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:55:34.517 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-19T12:55:34.517 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:55:34.517 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:55:34.517 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:34.517 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:55:34.517 INFO:tasks.workunit.client.0.trial144.stderr:188285: 0 2026-02-19T12:55:34.517 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:34.517 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:55:34.517 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:55:34.517 INFO:tasks.workunit.client.0.trial144.stderr:188285: 2026-02-19T12:55:33.515+0000 7fe327934a40 -1 Falling back to public interface 2026-02-19T12:55:34.517 INFO:tasks.workunit.client.0.trial144.stderr:188285: 2026-02-19T12:55:34.199+0000 7fe327934a40 -1 osd.9 84 log_to_monitors true 2026-02-19T12:55:34.518 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:55:34.518 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:34.518 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:55:34.518 INFO:tasks.workunit.client.0.trial144.stderr:188285: 1 2026-02-19T12:55:34.518 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:34.518 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:55:34.518 INFO:tasks.workunit.client.0.trial144.stderr:188285: osd.9 up in weight 1 up_from 88 up_thru 0 down_at 85 last_clean_interval [83,84) [v2:127.0.0.1:6842/3207639474,v1:127.0.0.1:6843/3207639474] [v2:127.0.0.1:6844/3207639474,v1:127.0.0.1:6845/3207639474] exists,up 19beb6b7-cd3d-4c41-8734-6c7b68c1a39d 2026-02-19T12:55:34.518 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:55:34.518 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:55:34.518 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:55:34.518 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:55:34.518 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:55:34.518 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:55:34.519 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:55:34.519 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:55:34.519 INFO:tasks.workunit.client.0.trial144.stderr:188285: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:55:35.068 INFO:tasks.workunit.client.0.trial144.stderr:188285: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:55:35.068 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:55:35.068 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:55:35.068 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:55:35.068 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:55:35.068 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:55:35.068 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:55:35.068 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:55:35.068 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:55:35.068 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:55:35.068 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:55:35.068 INFO:tasks.workunit.client.0.trial144.stderr:188285: 1 2026-02-19T12:55:35.068 INFO:tasks.workunit.client.0.trial144.stderr:188285: 2 2026-02-19T12:55:35.068 INFO:tasks.workunit.client.0.trial144.stderr:188285: 3 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: 4 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: 5 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: 6 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: 7 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: 8 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: 9' 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836509 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836509 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509' 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705692 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705692 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-38654705692' 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574876 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574876 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574876' 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=261993005073 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 261993005073 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574876 3-261993005073' 2026-02-19T12:55:35.069 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.074 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helperssh:1593: get_timeout_delays: echo true 2026-02-19T12:55:35.074 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:55:35.074 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:55:35.074 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:55:35.074 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:55:35.074 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:55:35.074 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:55:35.074 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:55:35.074 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:55:35.074 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:55:35.074 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:55:35.074 INFO:tasks.workunit.client.0.trial144.stderr:188283: 1 2026-02-19T12:55:35.074 INFO:tasks.workunit.client.0.trial144.stderr:188283: 2 2026-02-19T12:55:35.074 INFO:tasks.workunit.client.0.trial144.stderr:188283: 3 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: 4 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: 5 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: 6 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: 7 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: 8 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: 9' 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836510 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836510 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510' 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705693 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705693 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-38654705693' 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574877 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574877 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-38654705693 2-55834574877' 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=261993005074 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 261993005074 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-38654705693 2-55834574877 3-261993005074' 2026-02-19T12:55:35.075 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280538 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280538 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574876 3-261993005073 4-94489280538' 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=377957122050 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 377957122050 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574876 3-261993005073 4-94489280538 5-377957122050' 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018905 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018905 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574876 3-261993005073 4-94489280538 5-377957122050 6-128849018905' 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888088 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888088 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574876 3-261993005073 4-94489280538 5-377957122050 6-128849018905 7-146028888088' 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757271 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757271 2026-02-19T12:55:35.503 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574876 3-261993005073 4-94489280538 5-377957122050 6-128849018905 7-146028888088 8-163208757271' 2026-02-19T12:55:35.504 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.504 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:55:35.504 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=377957122050 2026-02-19T12:55:35.504 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 377957122050 2026-02-19T12:55:35.504 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574876 3-261993005073 4-94489280538 5-377957122050 6-128849018905 7-146028888088 8-163208757271 9-377957122050' 2026-02-19T12:55:35.504 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/us.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280539 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280539 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-38654705693 2-55834574877 3-261993005074 4-94489280539' 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=377957122051 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 377957122051 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-38654705693 2-55834574877 3-261993005074 4-94489280539 5-377957122051' 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018906 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018906 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-38654705693 2-55834574877 3-261993005074 4-94489280539 5-377957122051 6-128849018906' 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888089 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888089 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-38654705693 2-55834574877 3-261993005074 4-94489280539 5-377957122051 6-128849018906 7-146028888089' 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.509 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:55:35.510 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757272 2026-02-19T12:55:35.510 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757272 2026-02-19T12:55:35.510 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-38654705693 2-55834574877 3-261993005074 4-94489280539 5-377957122051 6-128849018906 7-146028888089 8-163208757272' 2026-02-19T12:55:35.510 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:35.510 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:55:35.510 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=377957122051 2026-02-19T12:55:35.510 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 377957122051 2026-02-19T12:55:35.510 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-38654705693 2-55834574877 3-261993005074 4-94489280539 5-377957122051 6-128849018906 7-146028888089 8-163208757272 9-377957122051' 2026-02-19T12:55:35.510 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:37.159 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836510 2026-02-19T12:55:37.160 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:37.160 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:55:37.160 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836510 2026-02-19T12:55:37.160 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:37.160 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836510 2026-02-19T12:55:37.160 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836510' 2026-02-19T12:55:37.161 INFO:tasks.workunit.client.0.trial144.stderr:188283: waiting osd.0 seq 21474836510 2026-02-19T12:55:37.161 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:37.161 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836508 -lt 21474836510 2026-02-19T12:55:37.161 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:55:37.161 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:55:37.161 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:37.162 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836510 -lt 21474836510 2026-02-19T12:55:37.162 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:37.162 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705693 2026-02-19T12:55:37.162 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:37.162 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:55:37.162 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705693 2026-02-19T12:55:37.163 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:37.163 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705693 2026-02-19T12:55:37.163 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705693' 2026-02-19T12:55:37.163 INFO:tasks.workunit.client.0.trial144.stderr:188283: waiting osd.1 seq 38654705693 2026-02-19T12:55:37.163 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:55:37.163 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705694 -lt 38654705693 2026-02-19T12:55:37.164 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:37.164 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574877 2026-02-19T12:55:37.164 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:37.164 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:55:37.164 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574877 2026-02-19T12:55:37.164 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:37.165 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574877 2026-02-19T12:55:37.165 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574877' 2026-02-19T12:55:37.165 INFO:tasks.workunit.client.0.trial144.stderr:188283: waiting osd.2 seq 55834574877 2026-02-19T12:55:37.165 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:55:37.165 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574877 -lt 55834574877 2026-02-19T12:55:37.166 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836509 2026-02-19T12:55:37.166 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:37.166 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:55:37.167 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836509 2026-02-19T12:55:37.167 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:37.167 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836509 2026-02-19T12:55:37.167 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836509' 2026-02-19T12:55:37.167 INFO:tasks.workunit.client.0.trial144.stderr:188285: waiting osd.0 seq 21474836509 2026-02-19T12:55:37.167 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:37.167 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836508 -lt 21474836509 2026-02-19T12:55:37.168 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:55:37.168 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:55:37.168 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:37.168 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836510 -lt 21474836509 2026-02-19T12:55:37.168 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:37.168 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705692 2026-02-19T12:55:37.168 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:37.168 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:55:37.168 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705692 2026-02-19T12:55:37.169 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:37.169 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705692 2026-02-19T12:55:37.169 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705692' 2026-02-19T12:55:37.169 INFO:tasks.workunit.client.0.trial144.stderr:188285: waiting osd.1 seq 38654705692 2026-02-19T12:55:37.169 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:55:37.169 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705694 -lt 38654705692 2026-02-19T12:55:37.169 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:37.170 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574876 2026-02-19T12:55:37.170 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:37.170 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:55:37.170 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574876 2026-02-19T12:55:37.170 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:37.170 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574876 2026-02-19T12:55:37.170 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574876' 2026-02-19T12:55:37.170 INFO:tasks.workunit.client.0.trial144.stderr:188285: waiting osd.2 seq 55834574876 2026-02-19T12:55:37.170 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:55:37.171 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574877 -lt 55834574876 2026-02-19T12:55:37.650 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flussh_pg_stats: for s in $seqs 2026-02-19T12:55:37.650 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-261993005074 2026-02-19T12:55:37.650 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:37.650 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:55:37.650 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-261993005074 2026-02-19T12:55:37.650 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:37.650 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=261993005074 2026-02-19T12:55:37.650 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 261993005074' 2026-02-19T12:55:37.650 INFO:tasks.workunit.client.0.trial144.stderr:188283: waiting osd.3 seq 261993005074 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 261993005075 -lt 261993005074 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280539 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-94489280539 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280539 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 94489280539' 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: waiting osd.4 seq 94489280539 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280539 -lt 94489280539 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-377957122051 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-377957122051 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=377957122051 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 377957122051' 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: waiting osd.5 seq 377957122051 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 377957122051 -lt 377957122051 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-128849018906 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:37.651 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:55:37.670 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standh_pg_stats: for s in $seqs 2026-02-19T12:55:37.670 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-261993005073 2026-02-19T12:55:37.670 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:37.670 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:55:37.670 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-261993005073 2026-02-19T12:55:37.670 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:37.670 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=261993005073 2026-02-19T12:55:37.670 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 261993005073' 2026-02-19T12:55:37.670 INFO:tasks.workunit.client.0.trial144.stderr:188285: waiting osd.3 seq 261993005073 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 261993005075 -lt 261993005073 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280538 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-94489280538 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280538 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 94489280538' 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: waiting osd.4 seq 94489280538 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280539 -lt 94489280538 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-377957122050 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-377957122050 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=377957122050 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 377957122050' 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: waiting osd.5 seq 377957122050 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 377957122051 -lt 377957122050 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-128849018905 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:37.671 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:55:38.140 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standaalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-128849018906 2026-02-19T12:55:38.140 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:38.140 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018906 2026-02-19T12:55:38.140 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 128849018906' 2026-02-19T12:55:38.140 INFO:tasks.workunit.client.0.trial144.stderr:188283: waiting osd.6 seq 128849018906 2026-02-19T12:55:38.140 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:55:38.141 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018906 -lt 128849018906 2026-02-19T12:55:38.141 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:38.141 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-146028888089 2026-02-19T12:55:38.141 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:38.141 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:55:38.141 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-146028888089 2026-02-19T12:55:38.141 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:38.141 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888089 2026-02-19T12:55:38.141 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 146028888089' 2026-02-19T12:55:38.141 INFO:tasks.workunit.client.0.trial144.stderr:188283: waiting osd.7 seq 146028888089 2026-02-19T12:55:38.141 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:55:38.141 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888089 -lt 146028888089 2026-02-19T12:55:38.142 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:38.142 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-163208757272 2026-02-19T12:55:38.142 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:38.142 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:55:38.142 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-163208757272 2026-02-19T12:55:38.142 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:38.142 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757272 2026-02-19T12:55:38.142 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 163208757272' 2026-02-19T12:55:38.142 INFO:tasks.workunit.client.0.trial144.stderr:188283: waiting osd.8 seq 163208757272 2026-02-19T12:55:38.142 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:55:38.142 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757273 -lt 163208757272 2026-02-19T12:55:38.142 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:38.143 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-377957122051 2026-02-19T12:55:38.143 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:38.143 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:55:38.143 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-377957122051 2026-02-19T12:55:38.143 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:38.143 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=377957122051 2026-02-19T12:55:38.170 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 slone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-128849018905 2026-02-19T12:55:38.170 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:38.170 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018905 2026-02-19T12:55:38.170 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 128849018905' 2026-02-19T12:55:38.170 INFO:tasks.workunit.client.0.trial144.stderr:188285: waiting osd.6 seq 128849018905 2026-02-19T12:55:38.171 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:55:38.171 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018906 -lt 128849018905 2026-02-19T12:55:38.171 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:38.171 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-146028888088 2026-02-19T12:55:38.171 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:38.171 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:55:38.171 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-146028888088 2026-02-19T12:55:38.172 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:38.172 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888088 2026-02-19T12:55:38.172 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 146028888088' 2026-02-19T12:55:38.172 INFO:tasks.workunit.client.0.trial144.stderr:188285: waiting osd.7 seq 146028888088 2026-02-19T12:55:38.172 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:55:38.172 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888089 -lt 146028888088 2026-02-19T12:55:38.172 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:38.172 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-163208757271 2026-02-19T12:55:38.172 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:38.172 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:55:38.172 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-163208757271 2026-02-19T12:55:38.172 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:38.173 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757271 2026-02-19T12:55:38.173 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 163208757271' 2026-02-19T12:55:38.173 INFO:tasks.workunit.client.0.trial144.stderr:188285: waiting osd.8 seq 163208757271 2026-02-19T12:55:38.173 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:55:38.173 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757273 -lt 163208757271 2026-02-19T12:55:38.173 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:38.173 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-377957122050 2026-02-19T12:55:38.173 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:38.173 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:55:38.173 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-377957122050 2026-02-19T12:55:38.173 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:38.173 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=377957122050 2026-02-19T12:55:38.848 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seeq 377957122051' 2026-02-19T12:55:38.848 INFO:tasks.workunit.client.0.trial144.stderr:188283: waiting osd.9 seq 377957122051 2026-02-19T12:55:38.848 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:55:38.848 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 377957122051 -lt 377957122051 2026-02-19T12:55:38.848 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:55:38.848 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:38.848 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:38.848 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:55:38.848 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:38.848 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:38.848 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:38.849 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:38.849 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:38.849 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:38.849 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:38.849 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:55:38.849 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:38.849 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:38.849 INFO:tasks.workunit.client.0.trial144.stderr:188283: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:38.849 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:55:38.849 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:55:38.849 INFO:tasks.workunit.client.0.trial144.stderr:188283: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:55:38.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:55:38.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:55:38.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 188281 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:q 377957122050' 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: waiting osd.9 seq 377957122050 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 377957122051 -lt 377957122050 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:55:38.901 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:38.902 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:38.902 INFO:tasks.workunit.client.0.trial144.stderr:188285: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:38.902 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:55:38.902 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:55:38.902 INFO:tasks.workunit.client.0.trial144.stderr:188285: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:55:38.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:55:38.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:55:38.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:55:38.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:55:38.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:272: corrupt_and_repair_two: return_code=0 2026-02-19T12:55:38.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:273: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:55:38.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:275: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:55:38.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:276: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:55:38.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:323: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 3 5 2026-02-19T12:55:38.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:245: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-19T12:55:38.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:246: corrupt_and_repair_two: local poolname=ecpool 2026-02-19T12:55:38.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:247: corrupt_and_repair_two: local first=3 2026-02-19T12:55:38.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_two: local second=5 2026-02-19T12:55:38.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_two: pids= 2026-02-19T12:55:38.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:55:38.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:55:38.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:55:38.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 191154"' 2026-02-19T12:55:38.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 191154' 2026-02-19T12:55:38.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:55:38.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:55:38.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:55:38.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:55:38.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 191156"' 2026-02-19T12:55:38.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 191156' 2026-02-19T12:55:38.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_two: wait_background pids 2026-02-19T12:55:38.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 191154 191156' 2026-02-19T12:55:38.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:55:38.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:55:38.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 191154 2026-02-19T12:55:38.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/191157: /' 2026-02-19T12:55:38.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:55:38.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/191160: /' 2026-02-19T12:55:39.562 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:55:39.563 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:55:39.563 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-19T12:55:39.563 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:55:39.563 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:55:39.563 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:55:39.563 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:55:39.563 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-19T12:55:39.563 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:55:39.563 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-19T12:55:39.564 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:55:39.564 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:55:39.564 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:55:39.564 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:55:39.564 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:55:39.564 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:55:39.564 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:55:39.564 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:55:39.564 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:55:39.564 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-19T12:55:39.564 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:55:39.564 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:55:39.565 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-19T12:55:39.565 INFO:tasks.workunit.client.0.trial144.stderr:191160: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-19T12:55:39.565 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-19T12:55:39.565 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:55:39.565 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:55:39.565 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-19T12:55:39.565 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:55:39.565 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:55:39.565 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:55:39.565 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:55:39.566 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:55:39.566 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:55:39.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-19T12:55:39.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-19T12:55:39.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:55:39.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:55:39.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:55:39.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:55:39.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:55:39.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:55:39.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:55:39.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:39.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:39.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:39.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:39.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:55:39.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:55:39.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:55:39.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:55:39.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:55:39.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:55:39.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:55:39.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:55:39.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:55:39.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-19T12:55:39.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-19T12:55:39.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: start osd.5 2026-02-19T12:55:39.569 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:55:39.574 INFO:tasks.workunit.client.0.trial144.stderr:1191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:55:39.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:55:39.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:55:39.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:55:39.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:55:39.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:55:39.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:55:39.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:55:39.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:55:39.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:55:39.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:55:39.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:55:39.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:55:39.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:55:39.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:55:39.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:55:39.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:55:39.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:55:39.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:55:39.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:55:39.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:55:39.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:55:39.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-19T12:55:39.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: remove 0#2:eb822e21:::SOMETHING:head# 2026-02-19T12:55:39.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:55:39.577 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:55:39.577 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:55:39.577 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:55:39.577 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:55:39.577 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:55:39.577 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:55:39.577 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:55:39.578 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:55:39.578 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:55:39.579 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:55:39.579 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:55:39.579 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:55:39.579 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:55:39.579 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:55:39.579 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:55:39.579 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:55:39.579 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:55:39.579 INFO:tasks.workunit.client.0.trial144.stderr:191157: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:55:39.579 INFO:tasks.workunit.client.0.trial144.stderr:191157: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:39.579 INFO:tasks.workunit.client.0.trial144.stderr:191157: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:39.579 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:39.580 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:39.580 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:55:39.580 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:55:39.580 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:55:39.580 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:55:39.580 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:55:39.580 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:55:39.580 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:55:39.580 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:55:39.580 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:55:39.580 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:55:39.580 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:55:39.580 INFO:tasks.workunit.client.0.trial144.stderr:191157: start osd.3 2026-02-19T12:55:39.581 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:55:41.052 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-19T12:55:41.052 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-19T12:55:41.052 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:55:41.052 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:55:41.053 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:55:41.053 INFO:tasks.workunit.client.0.trial144.stderr:191160: 2026-02-19T12:55:39.582+0000 7efcc4c1aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:41.053 INFO:tasks.workunit.client.0.trial144.stderr:191160: 2026-02-19T12:55:39.586+0000 7efcc4c1aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:41.053 INFO:tasks.workunit.client.0.trial144.stderr:191160: 2026-02-19T12:55:39.586+0000 7efcc4c1aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:41.053 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-19T12:55:41.053 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:55:41.053 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T12:55:41.053 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:55:41.053 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:55:41.053 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:41.053 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:55:41.054 INFO:tasks.workunit.client.0.trial144.stderr:191160: 0 2026-02-19T12:55:41.054 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:41.054 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:55:41.054 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:55:41.054 INFO:tasks.workunit.client.0.trial144.stderr:191160: 2026-02-19T12:55:40.034+0000 7efcc4c1aa40 -1 Falling back to public interface 2026-02-19T12:55:41.054 INFO:tasks.workunit.client.0.trial144.stderr:191160: 2026-02-19T12:55:40.714+0000 7efcc4c1aa40 -1 osd.5 89 log_to_monitors true 2026-02-19T12:55:41.054 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:55:41.054 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:41.054 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:55:41.054 INFO:tasks.workunit.client.0.trial144.stderr:191160: 1 2026-02-19T12:55:41.054 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:41.054 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:55:41.054 INFO:tasks.workunit.client.0.trial144.stderr:191160: osd.5 up in weight 1 up_from 93 up_thru 93 down_at 90 last_clean_interval [88,89) [v2:127.0.0.1:6826/3102573023,v1:127.0.0.1:6827/3102573023] [v2:127.0.0.1:6828/3102573023,v1:127.0.0.1:6829/3102573023] exists,up 2df300a0-d24a-42d1-b116-112bdb34ad84 2026-02-19T12:55:41.055 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:55:41.055 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:55:41.055 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:55:41.055 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:55:41.055 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:55:41.055 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:55:41.055 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:55:41.055 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:55:41.055 INFO:tasks.workunit.client.0.trial144.stderr:191160: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:55:41.059 INFO:tasks.workunit.client.0.trial144.stderr:191160: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: g91157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:55:41.059 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:55:41.059 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:55:41.059 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:55:41.059 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:55:41.059 INFO:tasks.workunit.client.0.trial144.stderr:191157: 2026-02-19T12:55:39.590+0000 7fa9f7c31a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:41.059 INFO:tasks.workunit.client.0.trial144.stderr:191157: 2026-02-19T12:55:39.590+0000 7fa9f7c31a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:41.059 INFO:tasks.workunit.client.0.trial144.stderr:191157: 2026-02-19T12:55:39.594+0000 7fa9f7c31a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:41.060 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:55:41.060 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:55:41.060 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:55:41.060 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:55:41.060 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:55:41.060 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:41.060 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:55:41.060 INFO:tasks.workunit.client.0.trial144.stderr:191157: 0 2026-02-19T12:55:41.060 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:41.060 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:55:41.060 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:55:41.060 INFO:tasks.workunit.client.0.trial144.stderr:191157: 2026-02-19T12:55:40.062+0000 7fa9f7c31a40 -1 Falling back to public interface 2026-02-19T12:55:41.060 INFO:tasks.workunit.client.0.trial144.stderr:191157: 2026-02-19T12:55:40.734+0000 7fa9f7c31a40 -1 osd.3 89 log_to_monitors true 2026-02-19T12:55:41.061 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:55:41.061 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:41.061 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:55:41.061 INFO:tasks.workunit.client.0.trial144.stderr:191157: 1 2026-02-19T12:55:41.061 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:41.061 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:55:41.061 INFO:tasks.workunit.client.0.trial144.stderr:191157: osd.3 up in weight 1 up_from 93 up_thru 93 down_at 90 last_clean_interval [61,89) [v2:127.0.0.1:6874/3632354893,v1:127.0.0.1:6875/3632354893] [v2:127.0.0.1:6876/3632354893,v1:127.0.0.1:6877/3632354893] exists,up 55da20d9-a5e9-4e5b-8bcc-8d8f03ecbf6d 2026-02-19T12:55:41.061 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:55:41.061 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:55:41.061 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:55:41.061 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:55:41.061 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:55:41.062 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:55:41.062 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:55:41.062 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:55:41.062 INFO:tasks.workunit.client.0.trial144.stderr:191157: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:55:41.615 INFO:tasks.workunit.client.0.trial144.stderr:191157: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:55:41.615 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:55:41.616 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:55:41.616 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:55:41.616 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:55:41.616 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:55:41.616 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:55:41.616 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:55:41.616 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:55:41.616 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:55:41.616 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:55:41.616 INFO:tasks.workunit.client.0.trial144.stderr:191160: 1 2026-02-19T12:55:41.616 INFO:tasks.workunit.client.0.trial144.stderr:191160: 2 2026-02-19T12:55:41.616 INFO:tasks.workunit.client.0.trial144.stderr:191160: 3 2026-02-19T12:55:41.617 INFO:tasks.workunit.client.0.trial144.stderr:191160: 4 2026-02-19T12:55:41.617 INFO:tasks.workunit.client.0.trial144.stderr:191160: 5 2026-02-19T12:55:41.617 INFO:tasks.workunit.client.0.trial144.stderr:191160: 6 2026-02-19T12:55:41.617 INFO:tasks.workunit.client.0.trial144.stderr:191160: 7 2026-02-19T12:55:41.617 INFO:tasks.workunit.client.0.trial144.stderr:191160: 8 2026-02-19T12:55:41.617 INFO:tasks.workunit.client.0.trial144.stderr:191160: 9' 2026-02-19T12:55:41.617 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:55:41.617 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:41.617 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:55:41.617 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836512 2026-02-19T12:55:41.617 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836512 2026-02-19T12:55:41.617 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836512' 2026-02-19T12:55:41.617 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:41.618 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:55:41.618 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705696 2026-02-19T12:55:41.618 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705696 2026-02-19T12:55:41.618 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836512 1-38654705696' 2026-02-19T12:55:41.618 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:41.618 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:55:41.618 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574879 2026-02-19T12:55:41.618 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574879 2026-02-19T12:55:41.618 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836512 1-38654705696 2-55834574879' 2026-02-19T12:55:41.618 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:41.618 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:55:41.618 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=399431958530 2026-02-19T12:55:41.619 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 399431958530 2026-02-19T12:55:41.619 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836512 1-38654705696 2-55834574879 3-399431958530' 2026-02-19T12:55:41.619 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:41.619 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258:et_timeout_delays: echo true 2026-02-19T12:55:41.619 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:55:41.619 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:55:41.620 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:55:41.620 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:55:41.620 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:55:41.620 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:55:41.620 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:55:41.620 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:55:41.620 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:55:41.620 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:55:41.620 INFO:tasks.workunit.client.0.trial144.stderr:191157: 1 2026-02-19T12:55:41.620 INFO:tasks.workunit.client.0.trial144.stderr:191157: 2 2026-02-19T12:55:41.620 INFO:tasks.workunit.client.0.trial144.stderr:191157: 3 2026-02-19T12:55:41.620 INFO:tasks.workunit.client.0.trial144.stderr:191157: 4 2026-02-19T12:55:41.621 INFO:tasks.workunit.client.0.trial144.stderr:191157: 5 2026-02-19T12:55:41.621 INFO:tasks.workunit.client.0.trial144.stderr:191157: 6 2026-02-19T12:55:41.621 INFO:tasks.workunit.client.0.trial144.stderr:191157: 7 2026-02-19T12:55:41.621 INFO:tasks.workunit.client.0.trial144.stderr:191157: 8 2026-02-19T12:55:41.621 INFO:tasks.workunit.client.0.trial144.stderr:191157: 9' 2026-02-19T12:55:41.621 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:55:41.621 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:41.621 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:55:41.621 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836513 2026-02-19T12:55:41.621 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836513 2026-02-19T12:55:41.621 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513' 2026-02-19T12:55:41.621 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:41.621 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:55:41.621 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705697 2026-02-19T12:55:41.622 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705697 2026-02-19T12:55:41.622 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-38654705697' 2026-02-19T12:55:41.622 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:41.622 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:55:41.622 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574880 2026-02-19T12:55:41.622 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574880 2026-02-19T12:55:41.622 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-38654705697 2-55834574880' 2026-02-19T12:55:41.622 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:41.622 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:55:41.622 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=399431958531 2026-02-19T12:55:41.622 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 399431958531 2026-02-19T12:55:41.622 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-38654705697 2-55834574880 3-399431958531' 2026-02-19T12:55:41.622 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:42.064 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:55:42.064 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280542 2026-02-19T12:55:42.064 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280542 2026-02-19T12:55:42.064 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836512 1-38654705696 2-55834574879 3-399431958530 4-94489280542' 2026-02-19T12:55:42.064 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:42.064 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:55:42.064 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=399431958530 2026-02-19T12:55:42.064 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 399431958530 2026-02-19T12:55:42.064 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836512 1-38654705696 2-55834574879 3-399431958530 4-94489280542 5-399431958530' 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018908 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018908 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836512 1-38654705696 2-55834574879 3-399431958530 4-94489280542 5-399431958530 6-128849018908' 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888092 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888092 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836512 1-38654705696 2-55834574879 3-399431958530 4-94489280542 5-399431958530 6-128849018908 7-146028888092' 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757276 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757276 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836512 1-38654705696 2-55834574879 3-399431958530 4-94489280542 5-399431958530 6-128849018908 7-146028888092 8-163208757276' 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=377957122054 2026-02-19T12:55:42.065 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 377957122054 2026-02-19T12:55:42.066 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836512 1-38654705696 2-55834574879 3-399431958530 4-94489280542 5-399431958530 6-128849018908 7-146028888092 8-163208757276 9-377957122054' 2026-02-19T12:55:42.066 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:42.066 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cep flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:55:42.066 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280543 2026-02-19T12:55:42.066 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280543 2026-02-19T12:55:42.066 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-38654705697 2-55834574880 3-399431958531 4-94489280543' 2026-02-19T12:55:42.066 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:42.066 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:55:42.066 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=399431958531 2026-02-19T12:55:42.066 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 399431958531 2026-02-19T12:55:42.066 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-38654705697 2-55834574880 3-399431958531 4-94489280543 5-399431958531' 2026-02-19T12:55:42.066 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:42.066 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018909 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018909 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-38654705697 2-55834574880 3-399431958531 4-94489280543 5-399431958531 6-128849018909' 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888091 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888091 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-38654705697 2-55834574880 3-399431958531 4-94489280543 5-399431958531 6-128849018909 7-146028888091' 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757275 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757275 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-38654705697 2-55834574880 3-399431958531 4-94489280543 5-399431958531 6-128849018909 7-146028888091 8-163208757275' 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=377957122053 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 377957122053 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-38654705697 2-55834574880 3-399431958531 4-94489280543 5-399431958531 6-128849018909 7-146028888091 8-163208757275 9-377957122053' 2026-02-19T12:55:42.067 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:42.565 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836512 2026-02-19T12:55:42.565 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:42.566 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:55:42.566 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836512 2026-02-19T12:55:42.566 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:42.566 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836512 2026-02-19T12:55:42.566 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836512' 2026-02-19T12:55:42.566 INFO:tasks.workunit.client.0.trial144.stderr:191160: waiting osd.0 seq 21474836512 2026-02-19T12:55:42.566 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:42.566 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836514 -lt 21474836512 2026-02-19T12:55:42.566 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:42.566 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705696 2026-02-19T12:55:42.566 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:42.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:55:42.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705696 2026-02-19T12:55:42.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:42.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705696 2026-02-19T12:55:42.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705696' 2026-02-19T12:55:42.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: waiting osd.1 seq 38654705696 2026-02-19T12:55:42.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:55:42.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705697 -lt 38654705696 2026-02-19T12:55:42.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:42.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574879 2026-02-19T12:55:42.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:42.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:55:42.567 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574879 2026-02-19T12:55:42.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:42.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574879 2026-02-19T12:55:42.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574879' 2026-02-19T12:55:42.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: waiting osd.2 seq 55834574879 2026-02-19T12:55:42.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:55:42.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574880 -lt 55834574879 2026-02-19T12:55:42.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:42.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-399431958530 2026-02-19T12:55:42.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:42.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:55:42.568 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-399431958530 2026-02-19T12:55:42.574 INFO:tasks.workunit.client.0.trial144.stderr:191htest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836513 2026-02-19T12:55:42.574 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:42.574 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:55:42.574 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836513 2026-02-19T12:55:42.574 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:42.574 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836513 2026-02-19T12:55:42.574 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836513' 2026-02-19T12:55:42.574 INFO:tasks.workunit.client.0.trial144.stderr:191157: waiting osd.0 seq 21474836513 2026-02-19T12:55:42.574 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:42.574 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836514 -lt 21474836513 2026-02-19T12:55:42.574 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:42.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705697 2026-02-19T12:55:42.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:42.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:55:42.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705697 2026-02-19T12:55:42.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:42.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705697 2026-02-19T12:55:42.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705697' 2026-02-19T12:55:42.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: waiting osd.1 seq 38654705697 2026-02-19T12:55:42.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:55:42.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705697 -lt 38654705697 2026-02-19T12:55:42.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:42.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574880 2026-02-19T12:55:42.575 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:42.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:55:42.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574880 2026-02-19T12:55:42.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:42.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574880 2026-02-19T12:55:42.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574880' 2026-02-19T12:55:42.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: waiting osd.2 seq 55834574880 2026-02-19T12:55:42.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:55:42.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574880 -lt 55834574880 2026-02-19T12:55:42.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:42.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-399431958531 2026-02-19T12:55:42.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:42.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:55:42.576 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-399431958531 2026-02-19T12:55:43.061 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:43.061 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=399431958530 2026-02-19T12:55:43.061 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 399431958530' 2026-02-19T12:55:43.061 INFO:tasks.workunit.client.0.trial144.stderr:191160: waiting osd.3 seq 399431958530 2026-02-19T12:55:43.061 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:55:43.061 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 399431958531 -lt 399431958530 2026-02-19T12:55:43.061 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:43.061 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280542 2026-02-19T12:55:43.061 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:43.061 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:55:43.061 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-94489280542 2026-02-19T12:55:43.061 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:43.061 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280542 2026-02-19T12:55:43.062 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 94489280542' 2026-02-19T12:55:43.062 INFO:tasks.workunit.client.0.trial144.stderr:191160: waiting osd.4 seq 94489280542 2026-02-19T12:55:43.062 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:55:43.062 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280543 -lt 94489280542 2026-02-19T12:55:43.062 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:43.062 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-399431958530 2026-02-19T12:55:43.062 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:43.062 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:55:43.062 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-399431958530 2026-02-19T12:55:43.062 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:43.062 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=399431958530 2026-02-19T12:55:43.062 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 399431958530' 2026-02-19T12:55:43.063 INFO:tasks.workunit.client.0.trial144.stderr:191160: waiting osd.5 seq 399431958530 2026-02-19T12:55:43.063 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:55:43.063 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 399431958531 -lt 399431958530 2026-02-19T12:55:43.063 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:43.063 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-128849018908 2026-02-19T12:55:43.063 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:43.063 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:55:43.063 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-128849018908 2026-02-19T12:55:43.063 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:43.063 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018908 2026-02-19T12:55:43.063 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 128849018908' 2026-02-19T12:55:43.063 INFO:tasks.workunit.client.0.trial144.stderr:191160: waiting osd.6 seq 128849018908 2026-02-19T12:55:43.069 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubun157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:43.069 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=399431958531 2026-02-19T12:55:43.069 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 399431958531' 2026-02-19T12:55:43.069 INFO:tasks.workunit.client.0.trial144.stderr:191157: waiting osd.3 seq 399431958531 2026-02-19T12:55:43.069 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:55:43.069 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 399431958531 -lt 399431958531 2026-02-19T12:55:43.069 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:43.069 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280543 2026-02-19T12:55:43.069 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:43.070 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:55:43.070 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-94489280543 2026-02-19T12:55:43.070 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:43.070 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280543 2026-02-19T12:55:43.070 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 94489280543' 2026-02-19T12:55:43.070 INFO:tasks.workunit.client.0.trial144.stderr:191157: waiting osd.4 seq 94489280543 2026-02-19T12:55:43.070 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:55:43.070 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280543 -lt 94489280543 2026-02-19T12:55:43.070 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:43.070 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-399431958531 2026-02-19T12:55:43.070 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:43.070 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:55:43.070 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-399431958531 2026-02-19T12:55:43.071 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:43.071 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=399431958531 2026-02-19T12:55:43.071 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 399431958531' 2026-02-19T12:55:43.071 INFO:tasks.workunit.client.0.trial144.stderr:191157: waiting osd.5 seq 399431958531 2026-02-19T12:55:43.071 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:55:43.071 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 399431958531 -lt 399431958531 2026-02-19T12:55:43.071 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:43.071 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-128849018909 2026-02-19T12:55:43.071 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:43.071 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:55:43.071 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-128849018909 2026-02-19T12:55:43.071 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:43.071 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018909 2026-02-19T12:55:43.072 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 128849018909' 2026-02-19T12:55:43.072 INFO:tasks.workunit.client.0.trial144.stderr:191157: waiting osd.6 seq 128849018909 2026-02-19T12:55:43.730 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:55:43.731 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018909 -lt 128849018909 2026-02-19T12:55:43.731 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:43.731 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-146028888091 2026-02-19T12:55:43.731 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:43.731 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:55:43.731 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-146028888091 2026-02-19T12:55:43.731 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:43.731 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888091 2026-02-19T12:55:43.731 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 146028888091' 2026-02-19T12:55:43.731 INFO:tasks.workunit.client.0.trial144.stderr:191157: waiting osd.7 seq 146028888091 2026-02-19T12:55:43.732 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:55:43.732 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888092 -lt 146028888091 2026-02-19T12:55:43.732 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:43.732 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-163208757275 2026-02-19T12:55:43.732 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:43.732 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:55:43.732 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-163208757275 2026-02-19T12:55:43.732 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:43.732 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757275 2026-02-19T12:55:43.732 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 163208757275' 2026-02-19T12:55:43.732 INFO:tasks.workunit.client.0.trial144.stderr:191157: waiting osd.8 seq 163208757275 2026-02-19T12:55:43.732 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:55:43.733 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757276 -lt 163208757275 2026-02-19T12:55:43.733 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:43.733 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-377957122053 2026-02-19T12:55:43.733 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:43.733 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:55:43.733 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-377957122053 2026-02-19T12:55:43.733 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:43.733 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=377957122053 2026-02-19T12:55:43.733 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 377957122053' 2026-02-19T12:55:43.733 INFO:tasks.workunit.client.0.trial144.stderr:191157: waiting osd.9 seq 377957122053 2026-02-19T12:55:43.733 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:55:43.733 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 377957122054 -lt 377957122053 2026-02-19T12:55:43.733 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:55:43.739 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceptu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:55:43.740 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018909 -lt 128849018908 2026-02-19T12:55:43.740 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:43.740 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-146028888092 2026-02-19T12:55:43.740 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:43.740 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:55:43.740 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-146028888092 2026-02-19T12:55:43.740 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:43.740 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888092 2026-02-19T12:55:43.740 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 146028888092' 2026-02-19T12:55:43.740 INFO:tasks.workunit.client.0.trial144.stderr:191160: waiting osd.7 seq 146028888092 2026-02-19T12:55:43.740 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:55:43.740 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888092 -lt 146028888092 2026-02-19T12:55:43.741 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:43.741 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-163208757276 2026-02-19T12:55:43.741 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:43.741 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:55:43.741 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-163208757276 2026-02-19T12:55:43.741 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:43.741 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757276 2026-02-19T12:55:43.741 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 163208757276' 2026-02-19T12:55:43.741 INFO:tasks.workunit.client.0.trial144.stderr:191160: waiting osd.8 seq 163208757276 2026-02-19T12:55:43.741 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:55:43.741 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757276 -lt 163208757276 2026-02-19T12:55:43.741 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:43.742 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-377957122054 2026-02-19T12:55:43.742 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:43.742 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:55:43.742 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-377957122054 2026-02-19T12:55:43.742 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:43.742 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=377957122054 2026-02-19T12:55:43.742 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 377957122054' 2026-02-19T12:55:43.742 INFO:tasks.workunit.client.0.trial144.stderr:191160: waiting osd.9 seq 377957122054 2026-02-19T12:55:43.742 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:55:43.742 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 377957122054 -lt 377957122054 2026-02-19T12:55:43.742 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:55:44.284 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:44.284 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:44.284 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:55:44.284 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:44.284 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:44.284 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:44.285 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:44.285 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:44.285 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:44.285 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:44.285 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:55:44.285 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:44.285 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:44.285 INFO:tasks.workunit.client.0.trial144.stderr:191157: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:44.285 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:55:44.285 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:55:44.285 INFO:tasks.workunit.client.0.trial144.stderr:191157: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:55:44.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:55:44.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:55:44.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 191156 2026-02-19T12:55:44.296 INFO:tasks.workunit.client.0.trial144.stderr:h-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:44.296 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:44.297 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:55:44.297 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:44.297 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:44.297 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:44.297 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:44.297 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:44.297 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:44.297 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:44.297 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:55:44.297 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:44.298 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:44.298 INFO:tasks.workunit.client.0.trial144.stderr:191160: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:44.298 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:55:44.298 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:55:44.298 INFO:tasks.workunit.client.0.trial144.stderr:191160: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:55:44.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:55:44.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:55:44.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:55:44.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:55:44.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:257: corrupt_and_repair_two: return_code=0 2026-02-19T12:55:44.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:258: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:55:44.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-19T12:55:44.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:55:44.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:55:44.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:55:44.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:55:44.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: local pg=2.0 2026-02-19T12:55:44.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:264: corrupt_and_repair_two: repair 2.0 2026-02-19T12:55:44.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:55:44.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:55:44.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:55:44.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:55:44.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:55:44.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:55:44.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:55:28.659912+0000 2026-02-19T12:55:44.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:55:44.768 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:55:44.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:55:28.659912+0000 2026-02-19T12:55:44.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:55:44.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:55:28.659912+0000 2026-02-19T12:55:44.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:55:44.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:55:44.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:55:44.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:55:44.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:55:44.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:55:44.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:55:44.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:55:44.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:55:28.659912+0000 '>' 2026-02-19T12:55:28.659912+0000 2026-02-19T12:55:44.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:55:45.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:55:45.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:55:45.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:55:45.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:55:45.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:55:45.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:55:45.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:55:46.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:55:28.659912+0000 '>' 2026-02-19T12:55:28.659912+0000 2026-02-19T12:55:46.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:55:47.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:55:47.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:55:47.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:55:47.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:55:47.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:55:47.100 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:55:47.100 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:55:47.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:55:44.793804+0000 '>' 2026-02-19T12:55:28.659912+0000 2026-02-19T12:55:47.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:55:47.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:268: corrupt_and_repair_two: pids= 2026-02-19T12:55:47.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:269: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:55:47.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:55:47.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:55:47.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 194470"' 2026-02-19T12:55:47.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 194470' 2026-02-19T12:55:47.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:270: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:55:47.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:55:47.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:55:47.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 194472"' 2026-02-19T12:55:47.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 194472' 2026-02-19T12:55:47.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:55:47.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:271: corrupt_and_repair_two: wait_background pids 2026-02-19T12:55:47.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 194470 194472' 2026-02-19T12:55:47.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:55:47.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:55:47.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 194470 2026-02-19T12:55:47.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/194473: /' 2026-02-19T12:55:47.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:55:47.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/194475: /' 2026-02-19T12:55:47.750 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:55:47.750 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:55:47.750 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:55:47.750 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:55:47.751 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:55:47.751 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:55:47.751 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:55:47.751 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:55:47.751 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:55:47.751 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:55:47.751 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:55:47.751 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:55:47.751 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:55:47.751 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:55:47.751 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:55:47.751 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:55:47.752 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:55:47.752 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:55:47.752 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:55:47.752 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:55:47.752 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:55:47.752 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:55:47.752 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-19T12:55:47.752 INFO:tasks.workunit.client.0.trial144.stderr:194473: Error getting attr on : 2.0s0_head,0#-4:00000000:::deep_scrub_2.0s0:head#, (61) No data available 2026-02-19T12:55:47.752 INFO:tasks.workunit.client.0.trial144.stderr:194473: Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-19T12:55:47.752 INFO:tasks.workunit.client.0.trial144.stderr:194473: _ 2026-02-19T12:55:47.752 INFO:tasks.workunit.client.0.trial144.stderr:194473: hinfo_key 2026-02-19T12:55:47.752 INFO:tasks.workunit.client.0.trial144.stderr:194473: snapset 2026-02-19T12:55:47.753 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:55:47.753 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:55:47.753 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:55:47.753 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:55:47.753 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:55:47.753 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:55:47.753 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:55:47.754 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:55:47.754 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:55:47.754 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:55:47.754 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:55:47.754 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:55:47.754 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:55:47.755 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:55:47.755 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:55:47.755 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:55:47.755 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:55:47.755 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:55:47.755 INFO:tasks.workunit.client.0.trial144.stderr:194473: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:55:47.755 INFO:tasks.workunit.client.0.trial144.stderr:194473: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:47.755 INFO:tasks.workunit.client.0.trial144.stderr:194473: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:47.755 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:47.755 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:47.755 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:55:47.755 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:55:47.755 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:55:47.756 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:55:47.756 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:55:47.756 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:55:47.756 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:55:47.756 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:55:47.756 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:55:47.756 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:55:47.756 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:55:47.756 INFO:tasks.workunit.client.0.trial144.stderr:194473: start osd.3 2026-02-19T12:55:47.762 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:55:47.762 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:55:47.763 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-19T12:55:47.763 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:55:47.763 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:55:47.763 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:55:47.763 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:55:47.764 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-19T12:55:47.764 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:55:47.764 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-19T12:55:47.764 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:55:47.764 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:55:47.764 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:55:47.765 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:55:47.765 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:55:47.765 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:55:47.765 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:55:47.765 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:55:47.765 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:55:47.766 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-19T12:55:47.766 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:55:47.766 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:55:47.766 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING list-attrs 2026-02-19T12:55:47.766 INFO:tasks.workunit.client.0.trial144.stderr:194475: _ 2026-02-19T12:55:47.766 INFO:tasks.workunit.client.0.trial144.stderr:194475: hinfo_key 2026-02-19T12:55:47.767 INFO:tasks.workunit.client.0.trial144.stderr:194475: snapset 2026-02-19T12:55:47.767 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-19T12:55:47.767 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:55:47.767 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:55:47.767 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-19T12:55:47.767 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:55:47.768 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:55:47.768 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:55:47.768 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:55:47.769 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:55:47.769 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:55:47.769 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-19T12:55:47.769 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-19T12:55:47.770 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:55:47.770 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:55:47.770 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:55:47.770 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:55:47.770 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:55:47.771 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:55:47.771 INFO:tasks.workunit.client.0.trial144.stderr:194475: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:55:47.771 INFO:tasks.workunit.client.0.trial144.stderr:194475: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:47.771 INFO:tasks.workunit.client.0.trial144.stderr:194475: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:47.771 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:47.771 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:47.772 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:55:47.772 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:55:47.772 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:55:47.772 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:55:47.772 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:55:47.772 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:55:47.773 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:55:47.773 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:55:47.773 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:55:47.773 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-19T12:55:47.773 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-19T12:55:47.773 INFO:tasks.workunit.client.0.trial144.stderr:194475: start osd.5 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recolog' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: 2026-02-19T12:55:47.770+0000 7fac7031ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: 2026-02-19T12:55:47.774+0000 7fac7031ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: 2026-02-19T12:55:47.774+0000 7fac7031ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: 0 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:49.236 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: 2026-02-19T12:55:48.162+0000 7fac7031ca40 -1 Falling back to public interface 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: 2026-02-19T12:55:48.802+0000 7fac7031ca40 -1 osd.3 94 log_to_monitors true 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: 1 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: osd.3 up in weight 1 up_from 98 up_thru 98 down_at 95 last_clean_interval [93,94) [v2:127.0.0.1:6826/370688266,v1:127.0.0.1:6827/370688266] [v2:127.0.0.1:6828/370688266,v1:127.0.0.1:6829/370688266] exists,up 55da20d9-a5e9-4e5b-8bcc-8d8f03ecbf6d 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:55:49.237 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:55:49.245 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays very_ops 2026-02-19T12:55:49.245 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-19T12:55:49.245 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-19T12:55:49.245 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:55:49.245 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:55:49.245 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:55:49.245 INFO:tasks.workunit.client.0.trial144.stderr:194475: 2026-02-19T12:55:47.778+0000 7fc4f6cfba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:49.245 INFO:tasks.workunit.client.0.trial144.stderr:194475: 2026-02-19T12:55:47.782+0000 7fc4f6cfba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:49.245 INFO:tasks.workunit.client.0.trial144.stderr:194475: 2026-02-19T12:55:47.782+0000 7fc4f6cfba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: 0 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: 2026-02-19T12:55:48.174+0000 7fc4f6cfba40 -1 Falling back to public interface 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: 2026-02-19T12:55:48.830+0000 7fc4f6cfba40 -1 osd.5 94 log_to_monitors true 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: 1 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: osd.5 up in weight 1 up_from 98 up_thru 98 down_at 95 last_clean_interval [93,94) [v2:127.0.0.1:6874/2174300495,v1:127.0.0.1:6875/2174300495] [v2:127.0.0.1:6876/2174300495,v1:127.0.0.1:6877/2174300495] exists,up 2df300a0-d24a-42d1-b116-112bdb34ad84 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:55:49.246 INFO:tasks.workunit.client.0.trial144.stderr:194475: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:55:49.799 INFO:tasks.workunit.client.0.trial144.stderr:194475: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:55:49.799 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:55:49.799 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:55:49.799 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:55:49.799 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:55:49.799 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:55:49.799 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:55:49.799 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: 1 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: 2 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: 3 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: 4 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: 5 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: 6 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: 7 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: 8 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: 9' 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836517 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836517 2026-02-19T12:55:49.800 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517' 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705700 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705700 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705700' 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574883 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574883 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705700 2-55834574883' 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=420906795010 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 420906795010 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705700 2-55834574883 3-420906795010' 2026-02-19T12:55:49.801 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:49.802 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper90 .1 2026-02-19T12:55:49.802 INFO:tasks.workunit.client.0.trial144.stderr:194473: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:55:49.802 INFO:tasks.workunit.client.0.trial144.stderr:194473: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:55:49.802 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:55:49.802 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:55:49.802 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:55:49.802 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:55:49.802 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:55:49.802 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:55:49.802 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:55:49.802 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:55:49.802 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:55:49.802 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:55:49.802 INFO:tasks.workunit.client.0.trial144.stderr:194473: 1 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: 2 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: 3 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: 4 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: 5 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: 6 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: 7 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: 8 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: 9' 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836516 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836516 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836516' 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705699 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705699 2026-02-19T12:55:49.803 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836516 1-38654705699' 2026-02-19T12:55:49.804 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:49.804 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:55:49.804 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574884 2026-02-19T12:55:49.804 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574884 2026-02-19T12:55:49.804 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574884' 2026-02-19T12:55:49.804 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:49.804 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:55:49.804 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=420906795011 2026-02-19T12:55:49.804 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 420906795011 2026-02-19T12:55:50.235 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-5583457488s.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:55:50.235 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280545 2026-02-19T12:55:50.235 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280545 2026-02-19T12:55:50.235 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705700 2-55834574883 3-420906795010 4-94489280545' 2026-02-19T12:55:50.235 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:50.235 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:55:50.236 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=420906795010 2026-02-19T12:55:50.236 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 420906795010 2026-02-19T12:55:50.236 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705700 2-55834574883 3-420906795010 4-94489280545 5-420906795010' 2026-02-19T12:55:50.236 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:50.236 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:55:50.236 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018912 2026-02-19T12:55:50.236 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018912 2026-02-19T12:55:50.236 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705700 2-55834574883 3-420906795010 4-94489280545 5-420906795010 6-128849018912' 2026-02-19T12:55:50.236 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:50.236 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:55:50.236 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888095 2026-02-19T12:55:50.236 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888095 2026-02-19T12:55:50.237 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705700 2-55834574883 3-420906795010 4-94489280545 5-420906795010 6-128849018912 7-146028888095' 2026-02-19T12:55:50.237 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:50.237 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:55:50.237 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757278 2026-02-19T12:55:50.237 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757278 2026-02-19T12:55:50.237 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705700 2-55834574883 3-420906795010 4-94489280545 5-420906795010 6-128849018912 7-146028888095 8-163208757278' 2026-02-19T12:55:50.237 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:50.237 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:55:50.237 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=377957122057 2026-02-19T12:55:50.237 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 377957122057 2026-02-19T12:55:50.237 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705700 2-55834574883 3-420906795010 4-94489280545 5-420906795010 6-128849018912 7-146028888095 8-163208757278 9-377957122057' 2026-02-19T12:55:50.238 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:50.238 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/4 3-420906795011' 2026-02-19T12:55:50.238 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:50.238 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:55:50.238 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280546 2026-02-19T12:55:50.238 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280546 2026-02-19T12:55:50.239 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574884 3-420906795011 4-94489280546' 2026-02-19T12:55:50.239 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:50.239 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:55:50.239 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=420906795011 2026-02-19T12:55:50.239 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 420906795011 2026-02-19T12:55:50.239 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574884 3-420906795011 4-94489280546 5-420906795011' 2026-02-19T12:55:50.239 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:50.239 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:55:50.239 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018913 2026-02-19T12:55:50.239 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018913 2026-02-19T12:55:50.239 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574884 3-420906795011 4-94489280546 5-420906795011 6-128849018913' 2026-02-19T12:55:50.239 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:50.240 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:55:50.240 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888096 2026-02-19T12:55:50.240 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888096 2026-02-19T12:55:50.240 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574884 3-420906795011 4-94489280546 5-420906795011 6-128849018913 7-146028888096' 2026-02-19T12:55:50.240 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:50.240 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:55:50.240 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757279 2026-02-19T12:55:50.240 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757279 2026-02-19T12:55:50.240 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574884 3-420906795011 4-94489280546 5-420906795011 6-128849018913 7-146028888096 8-163208757279' 2026-02-19T12:55:50.240 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:55:50.240 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:55:50.240 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=377957122058 2026-02-19T12:55:50.240 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 377957122058 2026-02-19T12:55:50.724 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574884 3-420906795011 4-94489280546 5-420906795011 6-128849018913 7-146028888096 8-163208757279 9-377957122058' 2026-02-19T12:55:50.724 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:50.724 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836516 2026-02-19T12:55:50.724 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:50.724 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:55:50.724 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836516 2026-02-19T12:55:50.724 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:50.724 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836516 2026-02-19T12:55:50.725 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836516' 2026-02-19T12:55:50.725 INFO:tasks.workunit.client.0.trial144.stderr:194473: waiting osd.0 seq 21474836516 2026-02-19T12:55:50.725 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:50.725 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836517 -lt 21474836516 2026-02-19T12:55:50.725 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:50.725 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705699 2026-02-19T12:55:50.725 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:50.725 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:55:50.725 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705699 2026-02-19T12:55:50.725 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:50.725 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705699 2026-02-19T12:55:50.725 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705699' 2026-02-19T12:55:50.726 INFO:tasks.workunit.client.0.trial144.stderr:194473: waiting osd.1 seq 38654705699 2026-02-19T12:55:50.726 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:55:50.726 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705700 -lt 38654705699 2026-02-19T12:55:50.726 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:50.726 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574884 2026-02-19T12:55:50.726 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:50.726 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:55:50.726 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574884 2026-02-19T12:55:50.726 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:50.726 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574884 2026-02-19T12:55:50.726 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574884' 2026-02-19T12:55:50.726 INFO:tasks.workunit.client.0.trial144.stderr:194473: waiting osd.2 seq 55834574884 2026-02-19T12:55:50.726 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:55:50.727 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574884 -lt 55834574884 2026-02-19T12:55:50.727 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:50.727 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-420906795011 2026-02-19T12:55:50.727 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:50.737 INFO:tasks.workunit.client.0.trial144.stderr:194473: /homeubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836517 2026-02-19T12:55:50.737 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:50.737 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:55:50.737 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836517 2026-02-19T12:55:50.737 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:50.737 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836517 2026-02-19T12:55:50.737 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836517' 2026-02-19T12:55:50.738 INFO:tasks.workunit.client.0.trial144.stderr:194475: waiting osd.0 seq 21474836517 2026-02-19T12:55:50.738 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:55:50.738 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836517 -lt 21474836517 2026-02-19T12:55:50.738 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:50.738 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705700 2026-02-19T12:55:50.738 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:50.738 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:55:50.738 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705700 2026-02-19T12:55:50.738 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:50.738 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705700 2026-02-19T12:55:50.738 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705700' 2026-02-19T12:55:50.738 INFO:tasks.workunit.client.0.trial144.stderr:194475: waiting osd.1 seq 38654705700 2026-02-19T12:55:50.738 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:55:50.739 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705700 -lt 38654705700 2026-02-19T12:55:50.739 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:50.739 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574883 2026-02-19T12:55:50.739 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:50.739 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:55:50.739 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574883 2026-02-19T12:55:50.739 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:50.739 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574883 2026-02-19T12:55:50.739 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574883' 2026-02-19T12:55:50.739 INFO:tasks.workunit.client.0.trial144.stderr:194475: waiting osd.2 seq 55834574883 2026-02-19T12:55:50.739 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:55:50.739 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574884 -lt 55834574883 2026-02-19T12:55:50.740 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:50.740 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-420906795010 2026-02-19T12:55:50.740 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:50.740 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:55:51.217 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-420906/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:55:51.217 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-420906795011 2026-02-19T12:55:51.217 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:51.217 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=420906795011 2026-02-19T12:55:51.217 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 420906795011' 2026-02-19T12:55:51.217 INFO:tasks.workunit.client.0.trial144.stderr:194473: waiting osd.3 seq 420906795011 2026-02-19T12:55:51.217 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:55:51.217 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 420906795011 -lt 420906795011 2026-02-19T12:55:51.218 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:51.218 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280546 2026-02-19T12:55:51.218 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:51.218 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:55:51.218 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-94489280546 2026-02-19T12:55:51.218 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:51.218 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280546 2026-02-19T12:55:51.218 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 94489280546' 2026-02-19T12:55:51.218 INFO:tasks.workunit.client.0.trial144.stderr:194473: waiting osd.4 seq 94489280546 2026-02-19T12:55:51.218 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:55:51.218 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280546 -lt 94489280546 2026-02-19T12:55:51.218 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:51.219 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-420906795011 2026-02-19T12:55:51.219 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:51.219 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:55:51.219 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-420906795011 2026-02-19T12:55:51.219 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:51.219 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=420906795011 2026-02-19T12:55:51.219 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 420906795011' 2026-02-19T12:55:51.219 INFO:tasks.workunit.client.0.trial144.stderr:194473: waiting osd.5 seq 420906795011 2026-02-19T12:55:51.219 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:55:51.219 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 420906795011 -lt 420906795011 2026-02-19T12:55:51.219 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:51.219 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-128849018913 2026-02-19T12:55:51.219 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:51.220 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:55:51.220 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-128849018913 2026-02-19T12:55:51.220 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:51.241 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats:795010 2026-02-19T12:55:51.241 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:51.241 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=420906795010 2026-02-19T12:55:51.241 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 420906795010' 2026-02-19T12:55:51.241 INFO:tasks.workunit.client.0.trial144.stderr:194475: waiting osd.3 seq 420906795010 2026-02-19T12:55:51.242 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:55:51.242 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 420906795011 -lt 420906795010 2026-02-19T12:55:51.242 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:51.242 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280545 2026-02-19T12:55:51.242 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:51.242 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:55:51.242 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-94489280545 2026-02-19T12:55:51.242 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:51.242 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280545 2026-02-19T12:55:51.242 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 94489280545' 2026-02-19T12:55:51.242 INFO:tasks.workunit.client.0.trial144.stderr:194475: waiting osd.4 seq 94489280545 2026-02-19T12:55:51.243 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:55:51.243 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280546 -lt 94489280545 2026-02-19T12:55:51.243 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:51.243 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-420906795010 2026-02-19T12:55:51.243 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:51.243 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:55:51.243 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-420906795010 2026-02-19T12:55:51.243 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:51.243 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=420906795010 2026-02-19T12:55:51.243 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 420906795010' 2026-02-19T12:55:51.244 INFO:tasks.workunit.client.0.trial144.stderr:194475: waiting osd.5 seq 420906795010 2026-02-19T12:55:51.244 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:55:51.244 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 420906795011 -lt 420906795010 2026-02-19T12:55:51.244 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:51.244 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-128849018912 2026-02-19T12:55:51.244 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:51.244 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:55:51.244 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-128849018912 2026-02-19T12:55:51.244 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:51.244 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018912 2026-02-19T12:55:51.244 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 128849018912' 2026-02-19T12:55:51.244 INFO:tasks.workunit.client.0.trial144.stderr:194475: waiting osd.6 seq 128849018912 2026-02-19T12:55:52.870 INFO:tasks.workunit.client.0.trial144.stderr:194475: / seq=128849018913 2026-02-19T12:55:52.870 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 128849018913' 2026-02-19T12:55:52.870 INFO:tasks.workunit.client.0.trial144.stderr:194473: waiting osd.6 seq 128849018913 2026-02-19T12:55:52.871 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:55:52.871 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018913 -lt 128849018913 2026-02-19T12:55:52.871 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:52.871 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-146028888096 2026-02-19T12:55:52.871 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:52.871 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:55:52.871 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-146028888096 2026-02-19T12:55:52.871 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:52.872 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888096 2026-02-19T12:55:52.872 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 146028888096' 2026-02-19T12:55:52.872 INFO:tasks.workunit.client.0.trial144.stderr:194473: waiting osd.7 seq 146028888096 2026-02-19T12:55:52.872 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:55:52.872 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888096 -lt 146028888096 2026-02-19T12:55:52.872 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:52.872 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-163208757279 2026-02-19T12:55:52.872 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:52.873 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:55:52.873 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-163208757279 2026-02-19T12:55:52.873 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:52.873 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757279 2026-02-19T12:55:52.873 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 163208757279' 2026-02-19T12:55:52.873 INFO:tasks.workunit.client.0.trial144.stderr:194473: waiting osd.8 seq 163208757279 2026-02-19T12:55:52.873 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:55:52.873 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757277 -lt 163208757279 2026-02-19T12:55:52.874 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:55:52.874 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:55:52.874 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:55:52.874 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757280 -lt 163208757279 2026-02-19T12:55:52.874 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:52.874 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-377957122058 2026-02-19T12:55:52.874 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:52.874 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:55:52.875 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-377957122058 2026-02-19T12:55:52.875 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:52.893 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:55:52.894 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018913 -lt 128849018912 2026-02-19T12:55:52.894 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:52.894 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-146028888095 2026-02-19T12:55:52.894 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:52.894 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:55:52.894 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-146028888095 2026-02-19T12:55:52.894 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:52.894 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888095 2026-02-19T12:55:52.894 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 146028888095' 2026-02-19T12:55:52.895 INFO:tasks.workunit.client.0.trial144.stderr:194475: waiting osd.7 seq 146028888095 2026-02-19T12:55:52.895 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:55:52.895 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888096 -lt 146028888095 2026-02-19T12:55:52.895 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:52.895 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-163208757278 2026-02-19T12:55:52.895 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:52.895 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:55:52.895 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-163208757278 2026-02-19T12:55:52.895 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:52.895 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757278 2026-02-19T12:55:52.895 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 163208757278' 2026-02-19T12:55:52.895 INFO:tasks.workunit.client.0.trial144.stderr:194475: waiting osd.8 seq 163208757278 2026-02-19T12:55:52.896 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:55:52.896 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757277 -lt 163208757278 2026-02-19T12:55:52.896 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:55:52.896 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:55:52.896 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:55:52.896 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757280 -lt 163208757278 2026-02-19T12:55:52.896 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:55:52.896 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-377957122057 2026-02-19T12:55:52.896 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:55:52.896 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:55:52.896 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-377957122057 2026-02-19T12:55:52.896 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:55:52.896 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=377957122057 2026-02-19T12:55:53.698 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=377957122058 2026-02-19T12:55:53.698 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 377957122058' 2026-02-19T12:55:53.698 INFO:tasks.workunit.client.0.trial144.stderr:194473: waiting osd.9 seq 377957122058 2026-02-19T12:55:53.699 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:55:53.699 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 377957122058 -lt 377957122058 2026-02-19T12:55:53.699 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:55:53.699 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:53.699 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:53.699 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:55:53.699 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:53.699 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:53.699 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:53.699 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:53.699 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:53.700 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:53.700 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:53.700 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:55:53.700 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:53.700 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:53.700 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:53.700 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:55:53.700 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-19T12:55:53.700 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T12:55:53.700 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-19T12:55:53.700 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:55:53.700 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T12:55:53.700 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:55:53.701 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:53.701 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:53.701 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:53.701 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:53.701 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:53.701 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:53.887 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone seq 377957122057' 2026-02-19T12:55:53.887 INFO:tasks.workunit.client.0.trial144.stderr:194475: waiting osd.9 seq 377957122057 2026-02-19T12:55:53.888 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:55:53.888 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 377957122058 -lt 377957122057 2026-02-19T12:55:53.888 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:55:53.888 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:53.888 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:53.888 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:55:53.888 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:53.888 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:53.888 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:53.888 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:53.888 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:53.889 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:53.889 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:53.889 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:55:53.889 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:53.889 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:53.889 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:53.889 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:55:53.889 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-19T12:55:53.889 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T12:55:53.889 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-19T12:55:53.889 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:55:53.889 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T12:55:53.890 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:55:53.890 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:53.890 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:53.890 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:53.890 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:53.890 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:53.890 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:53.890 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:54.623 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/c/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:54.623 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:55:54.624 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:54.624 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:54.624 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:54.624 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:55:54.624 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T12:55:54.624 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:55:54.624 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:55:54.624 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:55:54.624 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:55:54.624 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:55:54.625 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:55:54.625 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:55:54.625 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:55:54.625 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:55:54.625 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T12:55:54.625 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:55:54.625 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T12:55:54.625 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:55:54.625 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:54.625 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:54.625 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:54.625 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:54.625 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:54.626 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:54.626 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:54.626 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:55:54.626 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:54.626 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:54.821 INFO:tasks.workunit.client.0.trial144.stderr:194473: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:54.821 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:55:54.821 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:55:54.821 INFO:tasks.workunit.client.0.trial144.stderr:194473: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:55:54.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:55:54.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:55:54.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 194472 2026-02-19T12:55:54.823 INFO:tasks.workunit.client.0.trial144.stderr:lone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:55:54.824 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:54.824 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:54.824 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:54.824 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:55:54.824 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T12:55:54.824 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:55:54.824 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:55:54.824 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:55:54.824 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:55:54.824 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:55:54.824 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:55:54.825 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:55:54.825 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:55:54.825 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:55:54.825 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T12:55:54.825 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:55:54.825 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T12:55:54.825 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:55:54.825 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:55:54.825 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:55:54.825 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:55:54.825 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:55:54.825 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:55:54.826 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:55:54.826 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:55:54.826 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:55:54.826 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:55:54.826 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:55:54.826 INFO:tasks.workunit.client.0.trial144.stderr:194475: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:55:54.826 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:55:54.827 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:55:54.827 INFO:tasks.workunit.client.0.trial144.stderr:194475: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:55:54.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:55:54.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:55:54.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:55:54.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:55:54.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:272: corrupt_and_repair_two: return_code=0 2026-02-19T12:55:54.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:273: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:55:54.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:275: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:55:54.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:276: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:55:54.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T12:55:54.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:55:54.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:55:54.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:55:54.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:55:54.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:55:54.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:55:54.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:55:54.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:55:55.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:55:55.163 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:55:55.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:55:55.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:55:55.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:55:55.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:55:55.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:55:55.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:55:55.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:55:55.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:55:55.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:55:55.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:55:55.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:55:55.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:55:55.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:55:55.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:55.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:55.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:55:55.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:55:55.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:55:55.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T12:55:55.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T12:55:55.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T12:55:55.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T12:55:55.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:55:55.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:55:55.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:55:55.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:55:55.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:55:55.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:55:55.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:55:55.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:55:55.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:55:55.214 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:55:55.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:55:55.216 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:55:55.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:55:55.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:55:55.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:55:55.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:55:55.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:55:55.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:55:55.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:55:55.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:55:55.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:55:55.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:55:55.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:55:55.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:55.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:55.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:55:55.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:55:55.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:55:55.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T12:55:55.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:55:55.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:55.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:55.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T12:55:55.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:55:55.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T12:55:55.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:55:55.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T12:55:55.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_and_repair_lrc_overwrites td/osd-scrub-repair 2026-02-19T12:55:55.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:905: TEST_corrupt_and_repair_lrc_overwrites: '[' true = true ']' 2026-02-19T12:55:55.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:906: TEST_corrupt_and_repair_lrc_overwrites: corrupt_and_repair_lrc td/osd-scrub-repair true 2026-02-19T12:55:55.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:884: corrupt_and_repair_lrc: local dir=td/osd-scrub-repair 2026-02-19T12:55:55.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:885: corrupt_and_repair_lrc: local allow_overwrites=true 2026-02-19T12:55:55.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:886: corrupt_and_repair_lrc: local poolname=ecpool 2026-02-19T12:55:55.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:888: corrupt_and_repair_lrc: run_mon td/osd-scrub-repair a 2026-02-19T12:55:55.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T12:55:55.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T12:55:55.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T12:55:55.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T12:55:55.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T12:55:55.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T12:55:55.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T12:55:55.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:55:55.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:55:55.251 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:55:55.251 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:55.252 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:55.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:55.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T12:55:55.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T12:55:55.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T12:55:55.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:55:55.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:55:55.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T12:55:55.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T12:55:55.283 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:55:55.283 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:55:55.283 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:55:55.283 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:55:55.283 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:55.283 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:55.283 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:55:55.284 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:55:55.284 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T12:55:55.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T12:55:55.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:55:55.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:55:55.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T12:55:55.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T12:55:55.348 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:55:55.349 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:55:55.349 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:55:55.349 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:55:55.349 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:55.349 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:55.349 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:55:55.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:55:55.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T12:55:55.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:889: corrupt_and_repair_lrc: run_mgr td/osd-scrub-repair x 2026-02-19T12:55:55.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T12:55:55.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T12:55:55.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T12:55:55.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T12:55:55.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T12:55:55.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T12:55:55.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T12:55:55.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:55:55.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:55:55.502 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:55:55.502 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:55.502 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:55.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:55.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:55:55.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:55:55.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: seq 0 9 2026-02-19T12:55:55.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:55:55.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 0 2026-02-19T12:55:55.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:55:55.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:55:55.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T12:55:55.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:55:55.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:55:55.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:55:55.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:55:55.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:55:55.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:55:55.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:55:55.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:55:55.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:55:55.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:55:55.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:55:55.525 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:55:55.525 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:55:55.525 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:55:55.525 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:55:55.525 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:55.525 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:55.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:55.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:55.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:55:55.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:55:55.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:55:55.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:55:55.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:55:55.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:55:55.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:55:55.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:55:55.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:55:55.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:55:55.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:55:55.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:55:55.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:55:55.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e097429e-b1cc-4964-92f8-08c6b7f6d945 2026-02-19T12:55:55.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 e097429e-b1cc-4964-92f8-08c6b7f6d945' 2026-02-19T12:55:55.528 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 e097429e-b1cc-4964-92f8-08c6b7f6d945 2026-02-19T12:55:55.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:55:55.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBbCJdpH8sQIBAAoGrrHp6wROUnXMF9KVwx7w== 2026-02-19T12:55:55.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBbCJdpH8sQIBAAoGrrHp6wROUnXMF9KVwx7w=="}' 2026-02-19T12:55:55.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e097429e-b1cc-4964-92f8-08c6b7f6d945 -i td/osd-scrub-repair/0/new.json 2026-02-19T12:55:55.629 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:55:55.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T12:55:55.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBbCJdpH8sQIBAAoGrrHp6wROUnXMF9KVwx7w== --osd-uuid e097429e-b1cc-4964-92f8-08c6b7f6d945 2026-02-19T12:55:55.655 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:55.654+0000 7fb0f3964a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:55.657 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:55.654+0000 7fb0f3964a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:55.658 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:55.654+0000 7fb0f3964a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:55.658 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:55.658+0000 7fb0f3964a40 -1 bdev(0x560ee9088c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:55:55.658 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:55.658+0000 7fb0f3964a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T12:55:56.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T12:55:56.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:55:56.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T12:55:56.402 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:55:56.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:55:56.608 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:55:56.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T12:55:56.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:55:56.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:55:56.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:55:56.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:55:56.625 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:56.622+0000 7fba7beb4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:56.626 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:56.626+0000 7fba7beb4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:56.627 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:56.626+0000 7fba7beb4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:56.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T12:55:56.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:55:56.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:55:56.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:55:56.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:55:56.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:56.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:55:56.772 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:55:56.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:56.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:55:56.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:55:57.052 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:57.050+0000 7fba7beb4a40 -1 Falling back to public interface 2026-02-19T12:55:57.576 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:57.574+0000 7fba7beb4a40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:55:57.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:55:57.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:57.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:55:57.937 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:55:57.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:57.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:55:58.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:55:59.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:55:59.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:55:59.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:55:59.102 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:55:59.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:55:59.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:55:59.259 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1338972307,v1:127.0.0.1:6803/1338972307] [v2:127.0.0.1:6804/1338972307,v1:127.0.0.1:6805/1338972307] exists,up e097429e-b1cc-4964-92f8-08c6b7f6d945 2026-02-19T12:55:59.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 1 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:55:59.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:55:59.261 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:55:59.261 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:55:59.261 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:55:59.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:59.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:55:59.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:55:59.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:55:59.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:55:59.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:55:59.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:55:59.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:55:59.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:55:59.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:55:59.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:55:59.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:55:59.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:55:59.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:55:59.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:55:59.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=537a8097-bcc8-4651-b533-28587e51f2e4 2026-02-19T12:55:59.263 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 537a8097-bcc8-4651-b533-28587e51f2e4 2026-02-19T12:55:59.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 537a8097-bcc8-4651-b533-28587e51f2e4' 2026-02-19T12:55:59.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:55:59.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBfCJdp6/iOEBAADFJ7qaP22WH/w6foMa+2sw== 2026-02-19T12:55:59.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBfCJdp6/iOEBAADFJ7qaP22WH/w6foMa+2sw=="}' 2026-02-19T12:55:59.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 537a8097-bcc8-4651-b533-28587e51f2e4 -i td/osd-scrub-repair/1/new.json 2026-02-19T12:55:59.431 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:55:59.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T12:55:59.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBfCJdp6/iOEBAADFJ7qaP22WH/w6foMa+2sw== --osd-uuid 537a8097-bcc8-4651-b533-28587e51f2e4 2026-02-19T12:55:59.459 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:59.458+0000 7f7474dfaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:59.460 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:59.458+0000 7f7474dfaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:59.461 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:59.458+0000 7f7474dfaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:55:59.462 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:59.462+0000 7f7474dfaa40 -1 bdev(0x5561be4dcc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:55:59.462 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:55:59.462+0000 7f7474dfaa40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T12:56:00.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T12:56:00.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:56:00.141 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:56:00.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T12:56:00.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:56:00.336 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:56:00.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T12:56:00.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:56:00.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:56:00.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:56:00.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:56:00.351 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:00.350+0000 7f4acb160a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:00.353 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:00.350+0000 7f4acb160a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:00.354 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:00.354+0000 7f4acb160a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:00.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T12:56:00.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:56:00.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:56:00.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:56:00.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:56:00.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:00.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:56:00.500 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:56:00.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:00.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:56:00.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:00.775 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:00.774+0000 7f4acb160a40 -1 Falling back to public interface 2026-02-19T12:56:01.316 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:01.314+0000 7f4acb160a40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:56:01.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:01.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:01.661 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:56:01.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:56:01.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:01.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:56:01.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:02.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:02.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:02.838 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:56:02.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:56:02.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:02.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:56:02.996 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4278919536,v1:127.0.0.1:6811/4278919536] [v2:127.0.0.1:6812/4278919536,v1:127.0.0.1:6813/4278919536] exists,up 537a8097-bcc8-4651-b533-28587e51f2e4 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 2 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:56:02.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:56:02.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:56:02.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:56:02.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:56:02.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:56:02.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:56:02.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:56:02.998 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:56:02.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:56:03.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:56:03.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1dd1d362-f474-405b-b32f-e5122e34e313 2026-02-19T12:56:03.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 1dd1d362-f474-405b-b32f-e5122e34e313' 2026-02-19T12:56:03.002 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 1dd1d362-f474-405b-b32f-e5122e34e313 2026-02-19T12:56:03.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:56:03.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBjCJdphUvlABAAAZ3IiOfEUB0gIdBq6fN00g== 2026-02-19T12:56:03.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBjCJdphUvlABAAAZ3IiOfEUB0gIdBq6fN00g=="}' 2026-02-19T12:56:03.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1dd1d362-f474-405b-b32f-e5122e34e313 -i td/osd-scrub-repair/2/new.json 2026-02-19T12:56:03.178 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:56:03.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T12:56:03.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBjCJdphUvlABAAAZ3IiOfEUB0gIdBq6fN00g== --osd-uuid 1dd1d362-f474-405b-b32f-e5122e34e313 2026-02-19T12:56:03.207 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:03.206+0000 7f65d4d2ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:03.208 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:03.206+0000 7f65d4d2ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:03.209 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:03.206+0000 7f65d4d2ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:03.210 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:03.210+0000 7f65d4d2ea40 -1 bdev(0x557dd1080c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T12:56:03.210 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:03.210+0000 7f65d4d2ea40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T12:56:03.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T12:56:03.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:56:03.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T12:56:03.842 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T12:56:03.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:56:04.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T12:56:04.042 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T12:56:04.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:56:04.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:56:04.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:56:04.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:56:04.067 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:04.066+0000 7fd8660c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:04.069 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:04.066+0000 7fd8660c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:04.070 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:04.070+0000 7fd8660c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:04.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T12:56:04.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:56:04.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:56:04.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:56:04.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:56:04.200 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:56:04.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:04.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:56:04.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:04.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:56:04.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:04.479 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:04.478+0000 7fd8660c5a40 -1 Falling back to public interface 2026-02-19T12:56:05.065 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:05.062+0000 7fd8660c5a40 -1 osd.2 0 log_to_monitors true 2026-02-19T12:56:05.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:05.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:05.369 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:56:05.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:56:05.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:05.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:56:05.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:06.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:06.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:06.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:56:06.527 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:56:06.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:06.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:56:06.685 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/742037145,v1:127.0.0.1:6819/742037145] [v2:127.0.0.1:6820/742037145,v1:127.0.0.1:6821/742037145] exists,up 1dd1d362-f474-405b-b32f-e5122e34e313 2026-02-19T12:56:06.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:56:06.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:56:06.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:56:06.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:56:06.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 3 2026-02-19T12:56:06.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:56:06.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:56:06.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T12:56:06.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:56:06.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:56:06.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:56:06.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:56:06.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:56:06.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:56:06.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:56:06.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:56:06.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:56:06.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:56:06.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:56:06.689 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:56:06.690 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:56:06.690 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:56:06.690 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:56:06.690 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:56:06.690 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:56:06.690 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:06.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:06.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:56:06.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:56:06.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:56:06.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:56:06.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:56:06.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:56:06.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:56:06.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:56:06.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:56:06.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:56:06.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:56:06.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:56:06.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:56:06.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ef3f1d22-1abd-4877-a9fc-b7ff790fe73e 2026-02-19T12:56:06.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 ef3f1d22-1abd-4877-a9fc-b7ff790fe73e' 2026-02-19T12:56:06.692 INFO:tasks.workunit.client.0.trial144.stdout:add osd3 ef3f1d22-1abd-4877-a9fc-b7ff790fe73e 2026-02-19T12:56:06.693 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:56:06.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBmCJdp2XrjKRAAc43f8Ua1I4mO4NZ2Sq4wyw== 2026-02-19T12:56:06.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBmCJdp2XrjKRAAc43f8Ua1I4mO4NZ2Sq4wyw=="}' 2026-02-19T12:56:06.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ef3f1d22-1abd-4877-a9fc-b7ff790fe73e -i td/osd-scrub-repair/3/new.json 2026-02-19T12:56:06.864 INFO:tasks.workunit.client.0.trial144.stdout:3 2026-02-19T12:56:06.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-19T12:56:06.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBmCJdp2XrjKRAAc43f8Ua1I4mO4NZ2Sq4wyw== --osd-uuid ef3f1d22-1abd-4877-a9fc-b7ff790fe73e 2026-02-19T12:56:06.892 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:06.890+0000 7f98a460fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:06.894 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:06.890+0000 7f98a460fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:06.894 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:06.894+0000 7f98a460fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:06.895 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:06.894+0000 7f98a460fa40 -1 bdev(0x55e31cdf4c00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-19T12:56:06.895 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:06.894+0000 7f98a460fa40 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-19T12:56:07.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-19T12:56:07.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:56:07.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T12:56:07.502 INFO:tasks.workunit.client.0.trial144.stdout:adding osd3 key to auth repository 2026-02-19T12:56:07.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:56:07.707 INFO:tasks.workunit.client.0.trial144.stdout:start osd.3 2026-02-19T12:56:07.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T12:56:07.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:56:07.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:56:07.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:56:07.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:56:07.731 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:07.730+0000 7f6aed68ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:07.733 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:07.730+0000 7f6aed68ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:07.734 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:07.730+0000 7f6aed68ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:07.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T12:56:07.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:56:07.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:56:07.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:56:07.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:56:07.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:07.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:56:07.864 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:56:07.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:07.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:56:08.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:08.139 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:08.138+0000 7f6aed68ba40 -1 Falling back to public interface 2026-02-19T12:56:08.734 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:08.730+0000 7f6aed68ba40 -1 osd.3 0 log_to_monitors true 2026-02-19T12:56:09.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:09.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:09.022 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:56:09.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:56:09.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:09.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:56:09.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:10.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:10.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:10.180 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:56:10.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:56:10.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:10.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:56:10.339 INFO:tasks.workunit.client.0.trial144.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2832892637,v1:127.0.0.1:6827/2832892637] [v2:127.0.0.1:6828/2832892637,v1:127.0.0.1:6829/2832892637] exists,up ef3f1d22-1abd-4877-a9fc-b7ff790fe73e 2026-02-19T12:56:10.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:56:10.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:56:10.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:56:10.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:56:10.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 4 2026-02-19T12:56:10.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:56:10.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:56:10.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T12:56:10.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:56:10.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/4 2026-02-19T12:56:10.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:56:10.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:56:10.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:56:10.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:56:10.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/4' 2026-02-19T12:56:10.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/4/journal' 2026-02-19T12:56:10.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:56:10.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:56:10.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:56:10.343 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:56:10.343 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:56:10.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:56:10.344 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:56:10.344 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:56:10.344 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:56:10.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:10.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:10.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:56:10.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:56:10.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:56:10.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:56:10.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:56:10.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:56:10.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:56:10.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:56:10.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:56:10.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:56:10.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:56:10.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/4 2026-02-19T12:56:10.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:56:10.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4518a94a-af1a-49eb-9ad2-6a0db8db4385 2026-02-19T12:56:10.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 4518a94a-af1a-49eb-9ad2-6a0db8db4385' 2026-02-19T12:56:10.347 INFO:tasks.workunit.client.0.trial144.stdout:add osd4 4518a94a-af1a-49eb-9ad2-6a0db8db4385 2026-02-19T12:56:10.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:56:10.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBqCJdpclxWFRAAgjVPgB5kyVAleT8hxQETKg== 2026-02-19T12:56:10.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBqCJdpclxWFRAAgjVPgB5kyVAleT8hxQETKg=="}' 2026-02-19T12:56:10.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4518a94a-af1a-49eb-9ad2-6a0db8db4385 -i td/osd-scrub-repair/4/new.json 2026-02-19T12:56:10.522 INFO:tasks.workunit.client.0.trial144.stdout:4 2026-02-19T12:56:10.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/4/new.json 2026-02-19T12:56:10.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/4 --osd-journal=td/osd-scrub-repair/4/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBqCJdpclxWFRAAgjVPgB5kyVAleT8hxQETKg== --osd-uuid 4518a94a-af1a-49eb-9ad2-6a0db8db4385 2026-02-19T12:56:10.549 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:10.546+0000 7f0a84edca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:10.550 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:10.550+0000 7f0a84edca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:10.551 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:10.550+0000 7f0a84edca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:10.552 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:10.550+0000 7f0a84edca40 -1 bdev(0x55c9c5574c00 td/osd-scrub-repair/4/block) open stat got: (1) Operation not permitted 2026-02-19T12:56:10.552 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:10.550+0000 7f0a84edca40 -1 bluestore(td/osd-scrub-repair/4) _read_fsid unparsable uuid 2026-02-19T12:56:11.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/4/keyring 2026-02-19T12:56:11.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:56:11.166 INFO:tasks.workunit.client.0.trial144.stdout:adding osd4 key to auth repository 2026-02-19T12:56:11.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T12:56:11.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:56:11.366 INFO:tasks.workunit.client.0.trial144.stdout:start osd.4 2026-02-19T12:56:11.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T12:56:11.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/4 --osd-journal=td/osd-scrub-repair/4/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:56:11.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:56:11.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:56:11.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:56:11.383 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:11.382+0000 7f98c64c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:11.385 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:11.382+0000 7f98c64c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:11.386 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:11.386+0000 7f98c64c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:11.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T12:56:11.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:56:11.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T12:56:11.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:56:11.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:56:11.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:11.529 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:56:11.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:56:11.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:11.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T12:56:11.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:11.780 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:11.778+0000 7f98c64c7a40 -1 Falling back to public interface 2026-02-19T12:56:12.372 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:12.370+0000 7f98c64c7a40 -1 osd.4 0 log_to_monitors true 2026-02-19T12:56:12.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:12.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:12.686 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:56:12.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:56:12.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:12.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T12:56:12.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:13.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:13.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:13.846 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:56:13.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:56:13.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:13.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T12:56:14.004 INFO:tasks.workunit.client.0.trial144.stdout:osd.4 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1882069495,v1:127.0.0.1:6835/1882069495] [v2:127.0.0.1:6836/1882069495,v1:127.0.0.1:6837/1882069495] exists,up 4518a94a-af1a-49eb-9ad2-6a0db8db4385 2026-02-19T12:56:14.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:56:14.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:56:14.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:56:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:56:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 5 2026-02-19T12:56:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:56:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:56:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-19T12:56:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:56:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:56:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:56:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:56:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:56:14.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:56:14.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-19T12:56:14.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-19T12:56:14.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:56:14.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:56:14.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:56:14.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:56:14.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:56:14.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:56:14.009 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:56:14.009 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:56:14.009 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:56:14.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:14.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:56:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:56:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:56:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:56:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:56:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:56:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:56:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:56:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:56:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:56:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:56:14.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/5 2026-02-19T12:56:14.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:56:14.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f2b41730-2834-41c5-a57f-2a8a981fe2c0 2026-02-19T12:56:14.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 f2b41730-2834-41c5-a57f-2a8a981fe2c0' 2026-02-19T12:56:14.012 INFO:tasks.workunit.client.0.trial144.stdout:add osd5 f2b41730-2834-41c5-a57f-2a8a981fe2c0 2026-02-19T12:56:14.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:56:14.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBuCJdpri1ZARAA9YvGi0HFKj8h3GbOkXxZog== 2026-02-19T12:56:14.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBuCJdpri1ZARAA9YvGi0HFKj8h3GbOkXxZog=="}' 2026-02-19T12:56:14.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f2b41730-2834-41c5-a57f-2a8a981fe2c0 -i td/osd-scrub-repair/5/new.json 2026-02-19T12:56:14.186 INFO:tasks.workunit.client.0.trial144.stdout:5 2026-02-19T12:56:14.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/5/new.json 2026-02-19T12:56:14.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBuCJdpri1ZARAA9YvGi0HFKj8h3GbOkXxZog== --osd-uuid f2b41730-2834-41c5-a57f-2a8a981fe2c0 2026-02-19T12:56:14.213 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:14.210+0000 7f658980ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:14.214 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:14.214+0000 7f658980ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:14.215 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:14.214+0000 7f658980ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:14.216 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:14.214+0000 7f658980ba40 -1 bdev(0x55face5aac00 td/osd-scrub-repair/5/block) open stat got: (1) Operation not permitted 2026-02-19T12:56:14.216 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:14.214+0000 7f658980ba40 -1 bluestore(td/osd-scrub-repair/5) _read_fsid unparsable uuid 2026-02-19T12:56:14.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/5/keyring 2026-02-19T12:56:14.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:56:14.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-19T12:56:14.906 INFO:tasks.workunit.client.0.trial144.stdout:adding osd5 key to auth repository 2026-02-19T12:56:14.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:56:15.101 INFO:tasks.workunit.client.0.trial144.stdout:start osd.5 2026-02-19T12:56:15.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-19T12:56:15.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:56:15.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:56:15.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:56:15.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:56:15.126 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:15.126+0000 7fefb0445a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:15.128 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:15.126+0000 7fefb0445a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:15.129 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:15.126+0000 7fefb0445a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:15.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-19T12:56:15.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:56:15.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T12:56:15.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:56:15.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:56:15.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:15.262 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:56:15.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:56:15.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:15.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:56:15.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:15.492 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:15.490+0000 7fefb0445a40 -1 Falling back to public interface 2026-02-19T12:56:16.111 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:16.110+0000 7fefb0445a40 -1 osd.5 0 log_to_monitors true 2026-02-19T12:56:16.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:16.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:16.422 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:56:16.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:56:16.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:16.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:56:16.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:17.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:17.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:17.584 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:56:17.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:56:17.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:17.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:56:17.744 INFO:tasks.workunit.client.0.trial144.stdout:osd.5 up in weight 1 up_from 29 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1449103100,v1:127.0.0.1:6843/1449103100] [v2:127.0.0.1:6844/1449103100,v1:127.0.0.1:6845/1449103100] exists,up f2b41730-2834-41c5-a57f-2a8a981fe2c0 2026-02-19T12:56:17.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:56:17.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:56:17.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:56:17.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:56:17.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 6 2026-02-19T12:56:17.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:56:17.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:56:17.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-19T12:56:17.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:56:17.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/6 2026-02-19T12:56:17.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:56:17.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:56:17.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:56:17.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:56:17.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/6' 2026-02-19T12:56:17.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/6/journal' 2026-02-19T12:56:17.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:56:17.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:56:17.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:56:17.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:56:17.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:56:17.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:56:17.749 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:56:17.749 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:56:17.749 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:56:17.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:17.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:17.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:56:17.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:56:17.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:56:17.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:56:17.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:56:17.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:56:17.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:56:17.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:56:17.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:56:17.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:56:17.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:56:17.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/6 2026-02-19T12:56:17.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:56:17.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=56c220e4-c3b4-460a-952f-5e9bbce41f28 2026-02-19T12:56:17.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 56c220e4-c3b4-460a-952f-5e9bbce41f28' 2026-02-19T12:56:17.752 INFO:tasks.workunit.client.0.trial144.stdout:add osd6 56c220e4-c3b4-460a-952f-5e9bbce41f28 2026-02-19T12:56:17.752 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:56:17.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBxCJdpbz1zLRAA2Gbvrl6+nFV35naIVY5QTA== 2026-02-19T12:56:17.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBxCJdpbz1zLRAA2Gbvrl6+nFV35naIVY5QTA=="}' 2026-02-19T12:56:17.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 56c220e4-c3b4-460a-952f-5e9bbce41f28 -i td/osd-scrub-repair/6/new.json 2026-02-19T12:56:17.924 INFO:tasks.workunit.client.0.trial144.stdout:6 2026-02-19T12:56:17.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/6/new.json 2026-02-19T12:56:17.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/6 --osd-journal=td/osd-scrub-repair/6/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBxCJdpbz1zLRAA2Gbvrl6+nFV35naIVY5QTA== --osd-uuid 56c220e4-c3b4-460a-952f-5e9bbce41f28 2026-02-19T12:56:17.952 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:17.950+0000 7f9a436e5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:17.953 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:17.950+0000 7f9a436e5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:17.954 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:17.954+0000 7f9a436e5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:17.955 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:17.954+0000 7f9a436e5a40 -1 bdev(0x5555c6458c00 td/osd-scrub-repair/6/block) open stat got: (1) Operation not permitted 2026-02-19T12:56:17.955 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:17.954+0000 7f9a436e5a40 -1 bluestore(td/osd-scrub-repair/6) _read_fsid unparsable uuid 2026-02-19T12:56:18.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/6/keyring 2026-02-19T12:56:18.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:56:18.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-19T12:56:18.623 INFO:tasks.workunit.client.0.trial144.stdout:adding osd6 key to auth repository 2026-02-19T12:56:18.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:56:18.825 INFO:tasks.workunit.client.0.trial144.stdout:start osd.6 2026-02-19T12:56:18.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-19T12:56:18.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/6 --osd-journal=td/osd-scrub-repair/6/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:56:18.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:56:18.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:56:18.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:56:18.850 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:18.846+0000 7fdde6b22a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:18.852 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:18.850+0000 7fdde6b22a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:18.853 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:18.850+0000 7fdde6b22a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:18.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-19T12:56:18.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:56:18.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T12:56:18.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:56:18.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:56:18.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:18.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:56:18.989 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:56:18.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:18.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T12:56:19.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:19.276 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:19.274+0000 7fdde6b22a40 -1 Falling back to public interface 2026-02-19T12:56:19.860 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:19.858+0000 7fdde6b22a40 -1 osd.6 0 log_to_monitors true 2026-02-19T12:56:20.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:20.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:20.147 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:56:20.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:56:20.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:20.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T12:56:20.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:21.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:21.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:21.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:56:21.312 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:56:21.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:21.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T12:56:21.467 INFO:tasks.workunit.client.0.trial144.stdout:osd.6 up in weight 1 up_from 34 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/2324770082,v1:127.0.0.1:6851/2324770082] [v2:127.0.0.1:6852/2324770082,v1:127.0.0.1:6853/2324770082] exists,up 56c220e4-c3b4-460a-952f-5e9bbce41f28 2026-02-19T12:56:21.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:56:21.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:56:21.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:56:21.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:56:21.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 7 2026-02-19T12:56:21.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:56:21.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:56:21.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-19T12:56:21.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:56:21.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/7 2026-02-19T12:56:21.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:56:21.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:56:21.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:56:21.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:56:21.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/7' 2026-02-19T12:56:21.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/7/journal' 2026-02-19T12:56:21.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:56:21.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:56:21.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:56:21.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:56:21.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:56:21.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:56:21.469 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:56:21.469 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:56:21.469 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:56:21.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:21.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:21.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:56:21.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:56:21.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:56:21.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:56:21.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:56:21.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:56:21.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:56:21.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:56:21.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:56:21.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:56:21.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:56:21.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/7 2026-02-19T12:56:21.472 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:56:21.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d4e622e0-6352-4414-9956-20182e14870b 2026-02-19T12:56:21.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 d4e622e0-6352-4414-9956-20182e14870b' 2026-02-19T12:56:21.473 INFO:tasks.workunit.client.0.trial144.stdout:add osd7 d4e622e0-6352-4414-9956-20182e14870b 2026-02-19T12:56:21.473 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:56:21.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB1CJdpaOH2HBAAanEX1Oehr5HX8XFh8iIUkA== 2026-02-19T12:56:21.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB1CJdpaOH2HBAAanEX1Oehr5HX8XFh8iIUkA=="}' 2026-02-19T12:56:21.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d4e622e0-6352-4414-9956-20182e14870b -i td/osd-scrub-repair/7/new.json 2026-02-19T12:56:21.651 INFO:tasks.workunit.client.0.trial144.stdout:7 2026-02-19T12:56:21.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/7/new.json 2026-02-19T12:56:21.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/7 --osd-journal=td/osd-scrub-repair/7/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB1CJdpaOH2HBAAanEX1Oehr5HX8XFh8iIUkA== --osd-uuid d4e622e0-6352-4414-9956-20182e14870b 2026-02-19T12:56:21.689 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:21.686+0000 7f4ae554ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:21.690 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:21.690+0000 7f4ae554ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:21.691 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:21.690+0000 7f4ae554ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:21.692 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:21.690+0000 7f4ae554ca40 -1 bdev(0x559f4162ec00 td/osd-scrub-repair/7/block) open stat got: (1) Operation not permitted 2026-02-19T12:56:21.692 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:21.690+0000 7f4ae554ca40 -1 bluestore(td/osd-scrub-repair/7) _read_fsid unparsable uuid 2026-02-19T12:56:22.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/7/keyring 2026-02-19T12:56:22.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:56:22.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-19T12:56:22.379 INFO:tasks.workunit.client.0.trial144.stdout:adding osd7 key to auth repository 2026-02-19T12:56:22.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:56:22.586 INFO:tasks.workunit.client.0.trial144.stdout:start osd.7 2026-02-19T12:56:22.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-19T12:56:22.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/7 --osd-journal=td/osd-scrub-repair/7/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:56:22.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:56:22.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:56:22.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:56:22.610 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:22.610+0000 7fc6c9f73a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:22.612 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:22.610+0000 7fc6c9f73a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:22.613 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:22.610+0000 7fc6c9f73a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:22.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-19T12:56:22.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:56:22.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-19T12:56:22.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:56:22.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:56:22.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:22.744 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:56:22.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:56:22.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:22.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-19T12:56:22.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:23.040 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:23.038+0000 7fc6c9f73a40 -1 Falling back to public interface 2026-02-19T12:56:23.601 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:23.598+0000 7fc6c9f73a40 -1 osd.7 0 log_to_monitors true 2026-02-19T12:56:23.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:23.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:23.906 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:56:23.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:56:23.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:23.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-19T12:56:24.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:25.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:25.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:25.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:56:25.064 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:56:25.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:25.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-19T12:56:25.222 INFO:tasks.workunit.client.0.trial144.stdout:osd.7 up in weight 1 up_from 39 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/243305229,v1:127.0.0.1:6859/243305229] [v2:127.0.0.1:6860/243305229,v1:127.0.0.1:6861/243305229] exists,up d4e622e0-6352-4414-9956-20182e14870b 2026-02-19T12:56:25.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:56:25.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:56:25.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:56:25.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:56:25.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 8 2026-02-19T12:56:25.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:56:25.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:56:25.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-19T12:56:25.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:56:25.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/8 2026-02-19T12:56:25.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:56:25.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:56:25.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:56:25.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:56:25.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/8' 2026-02-19T12:56:25.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/8/journal' 2026-02-19T12:56:25.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:56:25.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:56:25.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:56:25.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:56:25.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:56:25.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:56:25.228 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:56:25.228 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:56:25.228 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:56:25.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:25.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:25.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:56:25.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:56:25.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:56:25.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:56:25.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:56:25.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:56:25.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:56:25.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:56:25.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:56:25.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:56:25.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:56:25.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/8 2026-02-19T12:56:25.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:56:25.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=68613b57-932f-408d-8705-00dee71ce97c 2026-02-19T12:56:25.230 INFO:tasks.workunit.client.0.trial144.stdout:add osd8 68613b57-932f-408d-8705-00dee71ce97c 2026-02-19T12:56:25.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 68613b57-932f-408d-8705-00dee71ce97c' 2026-02-19T12:56:25.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:56:25.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB5CJdpSV9qDhAA1C83eC8IEFZxosnqeTPkzw== 2026-02-19T12:56:25.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB5CJdpSV9qDhAA1C83eC8IEFZxosnqeTPkzw=="}' 2026-02-19T12:56:25.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 68613b57-932f-408d-8705-00dee71ce97c -i td/osd-scrub-repair/8/new.json 2026-02-19T12:56:25.405 INFO:tasks.workunit.client.0.trial144.stdout:8 2026-02-19T12:56:25.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/8/new.json 2026-02-19T12:56:25.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/8 --osd-journal=td/osd-scrub-repair/8/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB5CJdpSV9qDhAA1C83eC8IEFZxosnqeTPkzw== --osd-uuid 68613b57-932f-408d-8705-00dee71ce97c 2026-02-19T12:56:25.433 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:25.430+0000 7f14d7425a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:25.434 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:25.434+0000 7f14d7425a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:25.435 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:25.434+0000 7f14d7425a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:25.436 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:25.434+0000 7f14d7425a40 -1 bdev(0x564b7f04ec00 td/osd-scrub-repair/8/block) open stat got: (1) Operation not permitted 2026-02-19T12:56:25.436 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:25.434+0000 7f14d7425a40 -1 bluestore(td/osd-scrub-repair/8) _read_fsid unparsable uuid 2026-02-19T12:56:26.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/8/keyring 2026-02-19T12:56:26.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:56:26.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-19T12:56:26.083 INFO:tasks.workunit.client.0.trial144.stdout:adding osd8 key to auth repository 2026-02-19T12:56:26.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:56:26.292 INFO:tasks.workunit.client.0.trial144.stdout:start osd.8 2026-02-19T12:56:26.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-19T12:56:26.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/8 --osd-journal=td/osd-scrub-repair/8/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:56:26.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:56:26.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:56:26.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:56:26.317 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:26.314+0000 7f5bed69ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:26.318 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:26.318+0000 7f5bed69ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:26.319 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:26.318+0000 7f5bed69ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:26.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-19T12:56:26.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:56:26.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-19T12:56:26.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:56:26.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:56:26.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:26.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:56:26.448 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:56:26.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:26.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-19T12:56:26.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:26.712 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:26.710+0000 7f5bed69ea40 -1 Falling back to public interface 2026-02-19T12:56:27.340 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:27.337+0000 7f5bed69ea40 -1 osd.8 0 log_to_monitors true 2026-02-19T12:56:27.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:27.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:27.609 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:56:27.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:56:27.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:27.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-19T12:56:27.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:28.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:28.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:28.770 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:56:28.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:56:28.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:28.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-19T12:56:28.928 INFO:tasks.workunit.client.0.trial144.stdout:osd.8 up in weight 1 up_from 43 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/1866715393,v1:127.0.0.1:6867/1866715393] [v2:127.0.0.1:6868/1866715393,v1:127.0.0.1:6869/1866715393] exists,up 68613b57-932f-408d-8705-00dee71ce97c 2026-02-19T12:56:28.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:56:28.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:56:28.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:56:28.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:890: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-19T12:56:28.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:891: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 9 2026-02-19T12:56:28.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:56:28.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:56:28.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=9 2026-02-19T12:56:28.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:56:28.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:56:28.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:56:28.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:56:28.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:56:28.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:56:28.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-19T12:56:28.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-19T12:56:28.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:56:28.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:56:28.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:56:28.932 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:56:28.932 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:56:28.932 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:56:28.932 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:56:28.932 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:56:28.932 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:56:28.932 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:28.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:28.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:56:28.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:56:28.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:56:28.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:56:28.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:56:28.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:56:28.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:56:28.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:56:28.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:56:28.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:56:28.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:56:28.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/9 2026-02-19T12:56:28.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:56:28.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9f6223c2-19a3-4d20-afdc-51e479427486 2026-02-19T12:56:28.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd9 9f6223c2-19a3-4d20-afdc-51e479427486' 2026-02-19T12:56:28.935 INFO:tasks.workunit.client.0.trial144.stdout:add osd9 9f6223c2-19a3-4d20-afdc-51e479427486 2026-02-19T12:56:28.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:56:28.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB8CJdprvl1OBAAQdXC32Mj+ymAaAVf4pAWcw== 2026-02-19T12:56:28.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB8CJdprvl1OBAAQdXC32Mj+ymAaAVf4pAWcw=="}' 2026-02-19T12:56:28.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9f6223c2-19a3-4d20-afdc-51e479427486 -i td/osd-scrub-repair/9/new.json 2026-02-19T12:56:29.104 INFO:tasks.workunit.client.0.trial144.stdout:9 2026-02-19T12:56:29.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/9/new.json 2026-02-19T12:56:29.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 9 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB8CJdprvl1OBAAQdXC32Mj+ymAaAVf4pAWcw== --osd-uuid 9f6223c2-19a3-4d20-afdc-51e479427486 2026-02-19T12:56:29.132 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:29.129+0000 7f555f281a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:29.134 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:29.129+0000 7f555f281a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:29.135 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:29.133+0000 7f555f281a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:29.135 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:29.133+0000 7f555f281a40 -1 bdev(0x556462b86c00 td/osd-scrub-repair/9/block) open stat got: (1) Operation not permitted 2026-02-19T12:56:29.135 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:29.133+0000 7f555f281a40 -1 bluestore(td/osd-scrub-repair/9) _read_fsid unparsable uuid 2026-02-19T12:56:29.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/9/keyring 2026-02-19T12:56:29.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:56:29.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd9 key to auth repository 2026-02-19T12:56:29.786 INFO:tasks.workunit.client.0.trial144.stdout:adding osd9 key to auth repository 2026-02-19T12:56:29.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:56:29.983 INFO:tasks.workunit.client.0.trial144.stdout:start osd.9 2026-02-19T12:56:29.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.9 2026-02-19T12:56:29.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 9 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:56:29.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:56:29.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:56:29.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:56:30.007 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:30.005+0000 7f1e162c9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:30.009 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:30.005+0000 7f1e162c9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:30.010 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:30.009+0000 7f1e162c9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:30.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 9 2026-02-19T12:56:30.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:56:30.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-19T12:56:30.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:56:30.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:56:30.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:30.140 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:56:30.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:56:30.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:30.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:56:30.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:30.444 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:30.441+0000 7f1e162c9a40 -1 Falling back to public interface 2026-02-19T12:56:31.060 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:31.057+0000 7f1e162c9a40 -1 osd.9 0 log_to_monitors true 2026-02-19T12:56:31.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:31.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:31.304 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:56:31.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:56:31.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:31.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:56:31.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:32.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:32.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:32.464 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:56:32.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:56:32.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:32.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:56:32.626 INFO:tasks.workunit.client.0.trial144.stdout:osd.9 up in weight 1 up_from 47 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/2322524880,v1:127.0.0.1:6875/2322524880] [v2:127.0.0.1:6876/2322524880,v1:127.0.0.1:6877/2322524880] exists,up 9f6223c2-19a3-4d20-afdc-51e479427486 2026-02-19T12:56:32.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:56:32.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:56:32.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:56:32.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:893: corrupt_and_repair_lrc: create_rbd_pool 2026-02-19T12:56:32.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T12:56:32.775 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T12:56:32.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T12:56:32.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T12:56:33.005 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T12:56:33.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:56:34.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T12:56:34.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:894: corrupt_and_repair_lrc: wait_for_clean 2026-02-19T12:56:34.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:56:34.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:56:34.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:56:34.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:56:34.296 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:56:34.296 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:56:34.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:56:34.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:56:34.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:56:34.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:56:34.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:56:34.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:56:34.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:56:34.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:56:34.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:56:34.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:56:34.571 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:56:34.571 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:56:34.571 INFO:tasks.workunit.client.0.trial144.stderr:3 2026-02-19T12:56:34.571 INFO:tasks.workunit.client.0.trial144.stderr:4 2026-02-19T12:56:34.571 INFO:tasks.workunit.client.0.trial144.stderr:5 2026-02-19T12:56:34.571 INFO:tasks.workunit.client.0.trial144.stderr:6 2026-02-19T12:56:34.571 INFO:tasks.workunit.client.0.trial144.stderr:7 2026-02-19T12:56:34.571 INFO:tasks.workunit.client.0.trial144.stderr:8 2026-02-19T12:56:34.571 INFO:tasks.workunit.client.0.trial144.stderr:9' 2026-02-19T12:56:34.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:56:34.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:34.571 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:56:34.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836489 2026-02-19T12:56:34.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836489 2026-02-19T12:56:34.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489' 2026-02-19T12:56:34.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:34.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:56:34.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672968 2026-02-19T12:56:34.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672968 2026-02-19T12:56:34.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672968' 2026-02-19T12:56:34.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:34.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:56:34.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509447 2026-02-19T12:56:34.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509447 2026-02-19T12:56:34.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672968 2-64424509447' 2026-02-19T12:56:34.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:34.794 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:56:34.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=85899345926 2026-02-19T12:56:34.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 85899345926 2026-02-19T12:56:34.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672968 2-64424509447 3-85899345926' 2026-02-19T12:56:34.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:34.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:56:34.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182407 2026-02-19T12:56:34.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182407 2026-02-19T12:56:34.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672968 2-64424509447 3-85899345926 4-107374182407' 2026-02-19T12:56:34.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:34.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:56:35.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051589 2026-02-19T12:56:35.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051589 2026-02-19T12:56:35.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672968 2-64424509447 3-85899345926 4-107374182407 5-124554051589' 2026-02-19T12:56:35.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:35.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:56:35.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888068 2026-02-19T12:56:35.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888068 2026-02-19T12:56:35.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672968 2-64424509447 3-85899345926 4-107374182407 5-124554051589 6-146028888068' 2026-02-19T12:56:35.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:35.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:56:35.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724548 2026-02-19T12:56:35.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724548 2026-02-19T12:56:35.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672968 2-64424509447 3-85899345926 4-107374182407 5-124554051589 6-146028888068 7-167503724548' 2026-02-19T12:56:35.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:35.153 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:56:35.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593731 2026-02-19T12:56:35.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593731 2026-02-19T12:56:35.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672968 2-64424509447 3-85899345926 4-107374182407 5-124554051589 6-146028888068 7-167503724548 8-184683593731' 2026-02-19T12:56:35.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:35.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:56:35.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462914 2026-02-19T12:56:35.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462914 2026-02-19T12:56:35.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672968 2-64424509447 3-85899345926 4-107374182407 5-124554051589 6-146028888068 7-167503724548 8-184683593731 9-201863462914' 2026-02-19T12:56:35.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:35.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836489 2026-02-19T12:56:35.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:35.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:56:35.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836489 2026-02-19T12:56:35.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:35.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836489 2026-02-19T12:56:35.304 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836489 2026-02-19T12:56:35.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-19T12:56:35.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:56:35.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-19T12:56:35.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:56:36.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:56:36.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:56:36.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-19T12:56:36.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:36.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672968 2026-02-19T12:56:36.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:36.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:56:36.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672968 2026-02-19T12:56:36.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:36.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672968 2026-02-19T12:56:36.628 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672968 2026-02-19T12:56:36.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2026-02-19T12:56:36.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:56:36.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672968 -lt 42949672968 2026-02-19T12:56:36.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:36.788 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509447 2026-02-19T12:56:36.788 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:36.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:56:36.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509447 2026-02-19T12:56:36.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:36.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509447 2026-02-19T12:56:36.792 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 64424509447 2026-02-19T12:56:36.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509447' 2026-02-19T12:56:36.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:56:36.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509447 -lt 64424509447 2026-02-19T12:56:36.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:36.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-85899345926 2026-02-19T12:56:36.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:36.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:56:36.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-85899345926 2026-02-19T12:56:36.952 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:36.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=85899345926 2026-02-19T12:56:36.954 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.3 seq 85899345926 2026-02-19T12:56:36.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 85899345926' 2026-02-19T12:56:36.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:56:37.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 85899345927 -lt 85899345926 2026-02-19T12:56:37.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:37.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182407 2026-02-19T12:56:37.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:37.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:56:37.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-107374182407 2026-02-19T12:56:37.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:37.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182407 2026-02-19T12:56:37.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 107374182407' 2026-02-19T12:56:37.118 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.4 seq 107374182407 2026-02-19T12:56:37.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:56:37.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182407 -lt 107374182407 2026-02-19T12:56:37.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:37.287 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-124554051589 2026-02-19T12:56:37.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:37.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:56:37.289 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-124554051589 2026-02-19T12:56:37.290 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:37.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051589 2026-02-19T12:56:37.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 124554051589' 2026-02-19T12:56:37.291 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.5 seq 124554051589 2026-02-19T12:56:37.292 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:56:37.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051589 -lt 124554051589 2026-02-19T12:56:37.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:37.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-146028888068 2026-02-19T12:56:37.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:37.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:56:37.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-146028888068 2026-02-19T12:56:37.452 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:37.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888068 2026-02-19T12:56:37.453 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.6 seq 146028888068 2026-02-19T12:56:37.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 146028888068' 2026-02-19T12:56:37.454 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:56:37.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888068 -lt 146028888068 2026-02-19T12:56:37.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:37.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724548 2026-02-19T12:56:37.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:37.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:56:37.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-167503724548 2026-02-19T12:56:37.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:37.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724548 2026-02-19T12:56:37.623 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.7 seq 167503724548 2026-02-19T12:56:37.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 167503724548' 2026-02-19T12:56:37.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:56:37.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724548 -lt 167503724548 2026-02-19T12:56:37.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:37.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593731 2026-02-19T12:56:37.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:37.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:56:37.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-184683593731 2026-02-19T12:56:37.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:37.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593731 2026-02-19T12:56:37.783 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.8 seq 184683593731 2026-02-19T12:56:37.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 184683593731' 2026-02-19T12:56:37.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:56:37.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593731 -lt 184683593731 2026-02-19T12:56:37.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:37.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-201863462914 2026-02-19T12:56:37.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:37.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:56:37.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-201863462914 2026-02-19T12:56:37.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:37.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=201863462914 2026-02-19T12:56:37.952 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.9 seq 201863462914 2026-02-19T12:56:37.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 201863462914' 2026-02-19T12:56:37.953 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:56:38.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 201863462915 -lt 201863462914 2026-02-19T12:56:38.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:56:38.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:56:38.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:56:38.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:56:38.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:56:38.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:56:38.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:56:38.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:56:38.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:56:38.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:56:38.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:56:38.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:56:38.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:56:38.466 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:56:38.466 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:56:38.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:56:38.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:56:38.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:56:38.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:896: corrupt_and_repair_lrc: create_ec_pool ecpool true k=4 m=2 l=3 plugin=lrc 2026-02-19T12:56:38.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: create_ec_pool: local pool_name=ecpool 2026-02-19T12:56:38.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: create_ec_pool: shift 2026-02-19T12:56:38.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: create_ec_pool: local allow_overwrites=true 2026-02-19T12:56:38.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2525: create_ec_pool: shift 2026-02-19T12:56:38.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=4 m=2 l=3 plugin=lrc 2026-02-19T12:56:38.867 INFO:tasks.workunit.client.0.trial144.stderr:load: isa 2026-02-19T12:56:38.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-19T12:56:38.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-19T12:56:39.172 INFO:tasks.workunit.client.0.trial144.stderr:pool 'ecpool' created 2026-02-19T12:56:39.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:56:40.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: create_ec_pool: '[' true = true ']' 2026-02-19T12:56:40.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2532: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-19T12:56:40.383 INFO:tasks.workunit.client.0.trial144.stderr:set pool 2 allow_ec_overwrites to true 2026-02-19T12:56:40.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2535: create_ec_pool: wait_for_clean 2026-02-19T12:56:40.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:56:40.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:56:40.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:56:40.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:56:40.395 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:56:40.395 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:56:40.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:56:40.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:56:40.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:56:40.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:56:40.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:56:40.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:56:40.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:56:40.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:56:40.517 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:56:40.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:56:40.679 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:56:40.679 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:56:40.680 INFO:tasks.workunit.client.0.trial144.stderr:3 2026-02-19T12:56:40.680 INFO:tasks.workunit.client.0.trial144.stderr:4 2026-02-19T12:56:40.680 INFO:tasks.workunit.client.0.trial144.stderr:5 2026-02-19T12:56:40.680 INFO:tasks.workunit.client.0.trial144.stderr:6 2026-02-19T12:56:40.680 INFO:tasks.workunit.client.0.trial144.stderr:7 2026-02-19T12:56:40.680 INFO:tasks.workunit.client.0.trial144.stderr:8 2026-02-19T12:56:40.680 INFO:tasks.workunit.client.0.trial144.stderr:9' 2026-02-19T12:56:40.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:56:40.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:40.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:56:40.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-19T12:56:40.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-19T12:56:40.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491' 2026-02-19T12:56:40.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:40.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:56:40.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672970 2026-02-19T12:56:40.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672970 2026-02-19T12:56:40.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672970' 2026-02-19T12:56:40.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:40.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:56:40.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509449 2026-02-19T12:56:40.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509449 2026-02-19T12:56:40.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449' 2026-02-19T12:56:40.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:40.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:56:40.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=85899345929 2026-02-19T12:56:40.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 85899345929 2026-02-19T12:56:40.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449 3-85899345929' 2026-02-19T12:56:40.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:40.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:56:41.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182409 2026-02-19T12:56:41.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182409 2026-02-19T12:56:41.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449 3-85899345929 4-107374182409' 2026-02-19T12:56:41.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:41.049 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:56:41.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051591 2026-02-19T12:56:41.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051591 2026-02-19T12:56:41.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449 3-85899345929 4-107374182409 5-124554051591' 2026-02-19T12:56:41.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:41.122 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:56:41.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888071 2026-02-19T12:56:41.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888071 2026-02-19T12:56:41.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449 3-85899345929 4-107374182409 5-124554051591 6-146028888071' 2026-02-19T12:56:41.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:41.193 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:56:41.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724550 2026-02-19T12:56:41.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724550 2026-02-19T12:56:41.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449 3-85899345929 4-107374182409 5-124554051591 6-146028888071 7-167503724550' 2026-02-19T12:56:41.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:41.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:56:41.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593733 2026-02-19T12:56:41.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593733 2026-02-19T12:56:41.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449 3-85899345929 4-107374182409 5-124554051591 6-146028888071 7-167503724550 8-184683593733' 2026-02-19T12:56:41.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:41.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:56:41.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462916 2026-02-19T12:56:41.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462916 2026-02-19T12:56:41.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449 3-85899345929 4-107374182409 5-124554051591 6-146028888071 7-167503724550 8-184683593733 9-201863462916' 2026-02-19T12:56:41.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:41.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-19T12:56:41.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:41.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:56:41.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836491 2026-02-19T12:56:41.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:41.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836491 2026-02-19T12:56:41.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-19T12:56:41.415 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836491 2026-02-19T12:56:41.416 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:56:41.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T12:56:41.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:56:42.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:56:42.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:56:42.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-19T12:56:42.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:42.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672970 2026-02-19T12:56:42.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:42.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:56:42.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672970 2026-02-19T12:56:42.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:42.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672970 2026-02-19T12:56:42.735 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672970 2026-02-19T12:56:42.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-19T12:56:42.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:56:42.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672970 -lt 42949672970 2026-02-19T12:56:42.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:42.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509449 2026-02-19T12:56:42.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:42.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:56:42.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509449 2026-02-19T12:56:42.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:42.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509449 2026-02-19T12:56:42.900 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 64424509449 2026-02-19T12:56:42.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509449' 2026-02-19T12:56:42.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:56:43.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509450 -lt 64424509449 2026-02-19T12:56:43.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:43.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-85899345929 2026-02-19T12:56:43.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:43.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:56:43.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-85899345929 2026-02-19T12:56:43.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:43.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=85899345929 2026-02-19T12:56:43.072 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.3 seq 85899345929 2026-02-19T12:56:43.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 85899345929' 2026-02-19T12:56:43.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:56:43.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 85899345929 -lt 85899345929 2026-02-19T12:56:43.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:43.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182409 2026-02-19T12:56:43.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:43.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:56:43.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-107374182409 2026-02-19T12:56:43.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:43.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182409 2026-02-19T12:56:43.245 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.4 seq 107374182409 2026-02-19T12:56:43.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 107374182409' 2026-02-19T12:56:43.245 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:56:43.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182409 -lt 107374182409 2026-02-19T12:56:43.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:43.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-124554051591 2026-02-19T12:56:43.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:43.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:56:43.416 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-124554051591 2026-02-19T12:56:43.416 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:43.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051591 2026-02-19T12:56:43.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 124554051591' 2026-02-19T12:56:43.418 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.5 seq 124554051591 2026-02-19T12:56:43.418 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:56:43.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051591 -lt 124554051591 2026-02-19T12:56:43.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:43.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-146028888071 2026-02-19T12:56:43.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:43.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:56:43.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-146028888071 2026-02-19T12:56:43.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:43.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888071 2026-02-19T12:56:43.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 146028888071' 2026-02-19T12:56:43.593 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.6 seq 146028888071 2026-02-19T12:56:43.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:56:43.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888071 -lt 146028888071 2026-02-19T12:56:43.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:43.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724550 2026-02-19T12:56:43.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:43.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:56:43.763 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-167503724550 2026-02-19T12:56:43.763 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:43.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724550 2026-02-19T12:56:43.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 167503724550' 2026-02-19T12:56:43.765 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.7 seq 167503724550 2026-02-19T12:56:43.765 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:56:43.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724550 -lt 167503724550 2026-02-19T12:56:43.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:43.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593733 2026-02-19T12:56:43.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:43.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:56:43.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-184683593733 2026-02-19T12:56:43.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:43.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593733 2026-02-19T12:56:43.930 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.8 seq 184683593733 2026-02-19T12:56:43.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 184683593733' 2026-02-19T12:56:43.931 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:56:44.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593734 -lt 184683593733 2026-02-19T12:56:44.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:44.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-201863462916 2026-02-19T12:56:44.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:44.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:56:44.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-201863462916 2026-02-19T12:56:44.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:44.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=201863462916 2026-02-19T12:56:44.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 201863462916' 2026-02-19T12:56:44.099 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.9 seq 201863462916 2026-02-19T12:56:44.100 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:56:44.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 201863462917 -lt 201863462916 2026-02-19T12:56:44.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:56:44.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:56:44.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:56:44.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:56:44.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:56:44.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:56:44.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:56:44.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:56:44.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:56:44.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:56:44.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:56:44.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:56:44.615 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:56:44.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:56:44.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:56:44.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:56:44.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:56:44.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:56:44.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2536: create_ec_pool: return 0 2026-02-19T12:56:44.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:897: corrupt_and_repair_lrc: corrupt_and_repair_erasure_coded td/osd-scrub-repair ecpool 2026-02-19T12:56:44.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:308: corrupt_and_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-19T12:56:44.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:309: corrupt_and_repair_erasure_coded: local poolname=ecpool 2026-02-19T12:56:44.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:311: corrupt_and_repair_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-19T12:56:44.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T12:56:44.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T12:56:44.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-19T12:56:44.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T12:56:44.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T12:56:44.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T12:56:45.010 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T12:56:45.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T12:56:45.214 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T12:56:45.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T12:56:45.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T12:56:45.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-19T12:56:45.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:313: corrupt_and_repair_erasure_coded: get_primary ecpool SOMETHING 2026-02-19T12:56:45.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=ecpool 2026-02-19T12:56:45.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:56:45.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:56:45.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:56:45.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:313: corrupt_and_repair_erasure_coded: local primary=3 2026-02-19T12:56:45.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: get_osds ecpool SOMETHING 2026-02-19T12:56:45.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T12:56:45.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: sed -e s/3// 2026-02-19T12:56:45.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T12:56:45.415 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:56:45.415 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T12:56:45.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T12:56:45.572 INFO:tasks.workunit.client.0.trial144.stderr:5 2026-02-19T12:56:45.572 INFO:tasks.workunit.client.0.trial144.stderr:9 2026-02-19T12:56:45.573 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:56:45.573 INFO:tasks.workunit.client.0.trial144.stderr:6 2026-02-19T12:56:45.573 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:56:45.573 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:56:45.573 INFO:tasks.workunit.client.0.trial144.stderr:7' 2026-02-19T12:56:45.573 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 6 2 1 7 2026-02-19T12:56:45.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: osds=('5' '9' '0' '6' '2' '1' '7') 2026-02-19T12:56:45.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: corrupt_and_repair_erasure_coded: local -a osds 2026-02-19T12:56:45.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:315: corrupt_and_repair_erasure_coded: local not_primary_first=5 2026-02-19T12:56:45.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:316: corrupt_and_repair_erasure_coded: local not_primary_second=9 2026-02-19T12:56:45.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:319: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 3 2026-02-19T12:56:45.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:286: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-19T12:56:45.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:287: corrupt_and_repair_one: local poolname=ecpool 2026-02-19T12:56:45.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:288: corrupt_and_repair_one: local osd=3 2026-02-19T12:56:45.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:293: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:56:45.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:56:45.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:56:45.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:56:45.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:56:45.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:56:45.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:56:45.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:56:45.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:56:45.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:56:45.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:56:45.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:56:45.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:56:45.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:56:45.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:56:45.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:56:45.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:56:45.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:56:45.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:56:45.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:56:45.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:56:45.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:56:45.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:56:45.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-19T12:56:46.102 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#2:eb822e21:::SOMETHING:head# 2026-02-19T12:56:46.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:56:46.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:56:46.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:56:46.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:56:46.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:56:46.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:56:46.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:56:46.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:56:46.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:56:46.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:56:46.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:56:46.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:56:46.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:56:46.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:56:46.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:56:46.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:56:46.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:56:46.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:56:46.219 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:56:46.219 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:56:46.219 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:56:46.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:46.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:46.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:56:46.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:56:46.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:56:46.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:56:46.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:56:46.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:56:46.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:56:46.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:56:46.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:56:46.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:56:46.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:56:46.220 INFO:tasks.workunit.client.0.trial144.stderr:start osd.3 2026-02-19T12:56:46.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:56:46.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:56:46.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:56:46.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:56:46.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:56:46.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:56:46.236 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:46.233+0000 7ff5fdca0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:46.237 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:46.233+0000 7ff5fdca0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:46.238 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:46.233+0000 7ff5fdca0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:46.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:56:46.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:56:46.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:56:46.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:56:46.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:56:46.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:46.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:56:46.385 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:56:46.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:46.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:56:46.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:46.688 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:46.685+0000 7ff5fdca0a40 -1 Falling back to public interface 2026-02-19T12:56:47.309 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:47.305+0000 7ff5fdca0a40 -1 osd.3 64 log_to_monitors true 2026-02-19T12:56:47.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:56:47.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:47.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:56:47.544 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:56:47.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:47.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:56:47.695 INFO:tasks.workunit.client.0.trial144.stderr:osd.3 up in weight 1 up_from 68 up_thru 68 down_at 65 last_clean_interval [20,64) [v2:127.0.0.1:6826/2428301699,v1:127.0.0.1:6827/2428301699] [v2:127.0.0.1:6828/2428301699,v1:127.0.0.1:6829/2428301699] exists,up ef3f1d22-1abd-4877-a9fc-b7ff790fe73e 2026-02-19T12:56:47.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:56:47.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:56:47.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:56:47.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:56:47.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:56:47.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:56:47.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:56:47.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:56:47.697 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:56:47.697 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:56:47.697 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:56:47.697 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:56:47.697 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:56:47.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:56:47.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:56:47.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:56:47.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:56:47.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:56:47.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:56:47.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:56:47.976 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:56:47.977 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:56:47.977 INFO:tasks.workunit.client.0.trial144.stderr:3 2026-02-19T12:56:47.977 INFO:tasks.workunit.client.0.trial144.stderr:4 2026-02-19T12:56:47.977 INFO:tasks.workunit.client.0.trial144.stderr:5 2026-02-19T12:56:47.977 INFO:tasks.workunit.client.0.trial144.stderr:6 2026-02-19T12:56:47.977 INFO:tasks.workunit.client.0.trial144.stderr:7 2026-02-19T12:56:47.977 INFO:tasks.workunit.client.0.trial144.stderr:8 2026-02-19T12:56:47.977 INFO:tasks.workunit.client.0.trial144.stderr:9' 2026-02-19T12:56:47.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:56:47.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:47.978 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:56:48.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836493 2026-02-19T12:56:48.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836493 2026-02-19T12:56:48.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493' 2026-02-19T12:56:48.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:48.049 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:56:48.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672973 2026-02-19T12:56:48.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672973 2026-02-19T12:56:48.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-42949672973' 2026-02-19T12:56:48.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:48.122 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:56:48.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509452 2026-02-19T12:56:48.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509452 2026-02-19T12:56:48.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-42949672973 2-64424509452' 2026-02-19T12:56:48.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:48.193 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:56:48.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776130 2026-02-19T12:56:48.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776130 2026-02-19T12:56:48.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-42949672973 2-64424509452 3-292057776130' 2026-02-19T12:56:48.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:48.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:56:48.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182411 2026-02-19T12:56:48.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182411 2026-02-19T12:56:48.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-42949672973 2-64424509452 3-292057776130 4-107374182411' 2026-02-19T12:56:48.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:48.338 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:56:48.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051594 2026-02-19T12:56:48.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051594 2026-02-19T12:56:48.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-42949672973 2-64424509452 3-292057776130 4-107374182411 5-124554051594' 2026-02-19T12:56:48.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:48.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:56:48.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888073 2026-02-19T12:56:48.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888073 2026-02-19T12:56:48.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-42949672973 2-64424509452 3-292057776130 4-107374182411 5-124554051594 6-146028888073' 2026-02-19T12:56:48.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:48.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:56:48.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724552 2026-02-19T12:56:48.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724552 2026-02-19T12:56:48.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-42949672973 2-64424509452 3-292057776130 4-107374182411 5-124554051594 6-146028888073 7-167503724552' 2026-02-19T12:56:48.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:48.556 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:56:48.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593736 2026-02-19T12:56:48.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593736 2026-02-19T12:56:48.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-42949672973 2-64424509452 3-292057776130 4-107374182411 5-124554051594 6-146028888073 7-167503724552 8-184683593736' 2026-02-19T12:56:48.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:56:48.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:56:48.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462919 2026-02-19T12:56:48.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462919 2026-02-19T12:56:48.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-42949672973 2-64424509452 3-292057776130 4-107374182411 5-124554051594 6-146028888073 7-167503724552 8-184683593736 9-201863462919' 2026-02-19T12:56:48.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:48.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836493 2026-02-19T12:56:48.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:48.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:56:48.701 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836493 2026-02-19T12:56:48.701 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:48.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836493 2026-02-19T12:56:48.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-19T12:56:48.702 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836493 2026-02-19T12:56:48.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:56:48.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836492 -lt 21474836493 2026-02-19T12:56:48.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:56:49.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:56:49.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:56:50.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836492 -lt 21474836493 2026-02-19T12:56:50.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:56:51.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:56:51.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:56:51.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836494 -lt 21474836493 2026-02-19T12:56:51.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:51.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672973 2026-02-19T12:56:51.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:51.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:56:51.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672973 2026-02-19T12:56:51.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:51.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672973 2026-02-19T12:56:51.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672973' 2026-02-19T12:56:51.190 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 42949672973 2026-02-19T12:56:51.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:56:51.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672973 -lt 42949672973 2026-02-19T12:56:51.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:51.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509452 2026-02-19T12:56:51.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:51.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:56:51.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509452 2026-02-19T12:56:51.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:51.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509452 2026-02-19T12:56:51.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509452' 2026-02-19T12:56:51.351 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 64424509452 2026-02-19T12:56:51.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:56:51.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509452 -lt 64424509452 2026-02-19T12:56:51.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:51.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-292057776130 2026-02-19T12:56:51.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:51.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:56:51.517 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-292057776130 2026-02-19T12:56:51.517 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:51.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776130 2026-02-19T12:56:51.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 292057776130' 2026-02-19T12:56:51.519 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 292057776130 2026-02-19T12:56:51.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:56:51.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776130 -lt 292057776130 2026-02-19T12:56:51.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:51.679 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182411 2026-02-19T12:56:51.679 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:51.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:56:51.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-107374182411 2026-02-19T12:56:51.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:51.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182411 2026-02-19T12:56:51.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 107374182411' 2026-02-19T12:56:51.683 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.4 seq 107374182411 2026-02-19T12:56:51.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:56:51.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182412 -lt 107374182411 2026-02-19T12:56:51.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:51.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-124554051594 2026-02-19T12:56:51.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:51.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:56:51.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-124554051594 2026-02-19T12:56:51.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:51.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051594 2026-02-19T12:56:51.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 124554051594' 2026-02-19T12:56:51.848 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.5 seq 124554051594 2026-02-19T12:56:51.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:56:52.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051594 -lt 124554051594 2026-02-19T12:56:52.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:52.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-146028888073 2026-02-19T12:56:52.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:52.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:56:52.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-146028888073 2026-02-19T12:56:52.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:52.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888073 2026-02-19T12:56:52.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 146028888073' 2026-02-19T12:56:52.021 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.6 seq 146028888073 2026-02-19T12:56:52.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:56:52.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888074 -lt 146028888073 2026-02-19T12:56:52.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:52.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724552 2026-02-19T12:56:52.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:52.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:56:52.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-167503724552 2026-02-19T12:56:52.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:52.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724552 2026-02-19T12:56:52.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 167503724552' 2026-02-19T12:56:52.191 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.7 seq 167503724552 2026-02-19T12:56:52.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:56:52.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724553 -lt 167503724552 2026-02-19T12:56:52.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:52.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593736 2026-02-19T12:56:52.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:52.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:56:52.355 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-184683593736 2026-02-19T12:56:52.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:52.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593736 2026-02-19T12:56:52.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 184683593736' 2026-02-19T12:56:52.357 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.8 seq 184683593736 2026-02-19T12:56:52.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:56:52.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593736 -lt 184683593736 2026-02-19T12:56:52.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:56:52.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-201863462919 2026-02-19T12:56:52.525 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:56:52.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:56:52.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-201863462919 2026-02-19T12:56:52.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:56:52.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=201863462919 2026-02-19T12:56:52.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 201863462919' 2026-02-19T12:56:52.528 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.9 seq 201863462919 2026-02-19T12:56:52.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:56:52.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 201863462919 -lt 201863462919 2026-02-19T12:56:52.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:56:52.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:56:52.693 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:56:52.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:56:52.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:56:52.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:56:52.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:56:52.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:56:52.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:56:52.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:56:52.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:56:53.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:56:53.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:56:53.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:56:53.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:56:53.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:56:53.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-19T12:56:53.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T12:56:53.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-19T12:56:53.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:56:53.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T12:56:53.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:56:53.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:56:53.335 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:56:53.335 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:56:53.335 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:56:53.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:56:53.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:56:53.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:56:53.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:56:53.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:56:53.494 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:56:53.494 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:56:53.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:56:53.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T12:56:53.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:56:53.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:56:53.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:56:53.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:56:53.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:56:53.690 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:56:53.690 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:56:53.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:56:53.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:56:53.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T12:56:53.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:56:53.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T12:56:54.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:56:54.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:56:54.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:56:54.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:56:54.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:56:54.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:56:54.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:56:54.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:56:54.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:56:54.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:56:54.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:56:54.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:56:54.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:56:54.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:56:54.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:56:54.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-19T12:56:54.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:56:54.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:56:54.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:56:54.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:56:54.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: local pg=2.0 2026-02-19T12:56:54.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: corrupt_and_repair_one: repair 2.0 2026-02-19T12:56:54.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:56:54.601 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:56:54.601 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:56:54.601 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:56:54.601 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:56:54.601 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:56:54.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:56:39.173259+0000 2026-02-19T12:56:54.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:56:54.908 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:56:54.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:56:39.173259+0000 2026-02-19T12:56:54.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:56:54.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:56:39.173259+0000 2026-02-19T12:56:54.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:56:54.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:56:54.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:56:54.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:56:54.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:56:54.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:56:54.919 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:56:54.919 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:56:55.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:56:39.173259+0000 '>' 2026-02-19T12:56:39.173259+0000 2026-02-19T12:56:55.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:56:56.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:56:56.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:56:56.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:56:56.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:56:56.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:56:56.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:56:56.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:56:56.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:56:39.173259+0000 '>' 2026-02-19T12:56:39.173259+0000 2026-02-19T12:56:56.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:56:57.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:56:57.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:56:57.236 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:56:57.236 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:56:57.236 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:56:57.237 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:56:57.237 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:56:57.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:56:39.173259+0000 '>' 2026-02-19T12:56:39.173259+0000 2026-02-19T12:56:57.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:56:58.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:56:58.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:56:58.396 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:56:58.396 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:56:58.396 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:56:58.397 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:56:58.397 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:56:58.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:56:55.450552+0000 '>' 2026-02-19T12:56:39.173259+0000 2026-02-19T12:56:58.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:56:58.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:302: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:56:58.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:56:58.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:56:58.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-19T12:56:58.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:56:58.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:56:58.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:56:58.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:56:58.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-19T12:56:58.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:56:58.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-19T12:56:58.553 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:56:58.553 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:56:58.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:56:58.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:56:58.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:56:58.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:56:58.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:56:58.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:56:58.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:56:58.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-19T12:56:58.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:56:58.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:56:58.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-19T12:56:58.944 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:56:58.944 INFO:tasks.workunit.client.0.trial144.stdout:hinfo_key 2026-02-19T12:56:58.944 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:56:59.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:56:59.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:56:59.044 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:56:59.044 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:56:59.044 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:56:59.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:59.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:56:59.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:56:59.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:56:59.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:56:59.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:56:59.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:56:59.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:56:59.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:56:59.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:56:59.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:56:59.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-19T12:56:59.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-19T12:56:59.045 INFO:tasks.workunit.client.0.trial144.stderr:start osd.9 2026-02-19T12:56:59.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:56:59.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-19T12:56:59.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-19T12:56:59.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:56:59.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:56:59.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:56:59.068 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:59.065+0000 7fe4f6366a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:59.069 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:59.065+0000 7fe4f6366a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:59.070 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:59.069+0000 7fe4f6366a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:56:59.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-19T12:56:59.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:56:59.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-19T12:56:59.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:56:59.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:56:59.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:56:59.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:56:59.210 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:56:59.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:56:59.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:56:59.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:56:59.420 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:56:59.417+0000 7fe4f6366a40 -1 Falling back to public interface 2026-02-19T12:57:00.058 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:57:00.057+0000 7fe4f6366a40 -1 osd.9 70 log_to_monitors true 2026-02-19T12:57:00.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:57:00.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:00.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:57:00.368 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:57:00.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:00.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:57:00.522 INFO:tasks.workunit.client.0.trial144.stderr:osd.9 up in weight 1 up_from 73 up_thru 0 down_at 70 last_clean_interval [47,70) [v2:127.0.0.1:6874/1694857941,v1:127.0.0.1:6875/1694857941] [v2:127.0.0.1:6876/1694857941,v1:127.0.0.1:6877/1694857941] exists,up 9f6223c2-19a3-4d20-afdc-51e479427486 2026-02-19T12:57:00.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:57:00.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:57:00.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:57:00.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:57:00.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:57:00.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:57:00.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:57:00.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:57:00.523 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:57:00.523 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:57:00.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:57:00.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:57:00.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:57:00.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:57:00.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:57:00.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:57:00.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:57:00.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:57:00.628 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:57:00.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:57:00.790 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:57:00.790 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:57:00.790 INFO:tasks.workunit.client.0.trial144.stderr:3 2026-02-19T12:57:00.790 INFO:tasks.workunit.client.0.trial144.stderr:4 2026-02-19T12:57:00.790 INFO:tasks.workunit.client.0.trial144.stderr:5 2026-02-19T12:57:00.790 INFO:tasks.workunit.client.0.trial144.stderr:6 2026-02-19T12:57:00.790 INFO:tasks.workunit.client.0.trial144.stderr:7 2026-02-19T12:57:00.790 INFO:tasks.workunit.client.0.trial144.stderr:8 2026-02-19T12:57:00.790 INFO:tasks.workunit.client.0.trial144.stderr:9' 2026-02-19T12:57:00.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:57:00.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:00.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:57:00.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836497 2026-02-19T12:57:00.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836497 2026-02-19T12:57:00.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497' 2026-02-19T12:57:00.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:00.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:57:00.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672976 2026-02-19T12:57:00.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672976 2026-02-19T12:57:00.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-42949672976' 2026-02-19T12:57:00.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:00.934 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:57:01.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509455 2026-02-19T12:57:01.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509455 2026-02-19T12:57:01.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-42949672976 2-64424509455' 2026-02-19T12:57:01.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:01.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:57:01.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776133 2026-02-19T12:57:01.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776133 2026-02-19T12:57:01.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-42949672976 2-64424509455 3-292057776133' 2026-02-19T12:57:01.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:01.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:57:01.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182415 2026-02-19T12:57:01.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182415 2026-02-19T12:57:01.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-42949672976 2-64424509455 3-292057776133 4-107374182415' 2026-02-19T12:57:01.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:01.153 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:57:01.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051597 2026-02-19T12:57:01.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051597 2026-02-19T12:57:01.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-42949672976 2-64424509455 3-292057776133 4-107374182415 5-124554051597' 2026-02-19T12:57:01.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:01.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:57:01.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888077 2026-02-19T12:57:01.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888077 2026-02-19T12:57:01.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-42949672976 2-64424509455 3-292057776133 4-107374182415 5-124554051597 6-146028888077' 2026-02-19T12:57:01.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:01.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:57:01.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724556 2026-02-19T12:57:01.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724556 2026-02-19T12:57:01.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-42949672976 2-64424509455 3-292057776133 4-107374182415 5-124554051597 6-146028888077 7-167503724556' 2026-02-19T12:57:01.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:01.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:57:01.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593739 2026-02-19T12:57:01.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593739 2026-02-19T12:57:01.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-42949672976 2-64424509455 3-292057776133 4-107374182415 5-124554051597 6-146028888077 7-167503724556 8-184683593739' 2026-02-19T12:57:01.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:01.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:57:01.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=313532612610 2026-02-19T12:57:01.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 313532612610 2026-02-19T12:57:01.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-42949672976 2-64424509455 3-292057776133 4-107374182415 5-124554051597 6-146028888077 7-167503724556 8-184683593739 9-313532612610' 2026-02-19T12:57:01.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:01.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836497 2026-02-19T12:57:01.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:01.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:57:01.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836497 2026-02-19T12:57:01.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:01.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836497 2026-02-19T12:57:01.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2026-02-19T12:57:01.521 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836497 2026-02-19T12:57:01.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:01.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836496 -lt 21474836497 2026-02-19T12:57:01.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:57:02.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:57:02.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:02.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836497 -lt 21474836497 2026-02-19T12:57:02.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:02.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672976 2026-02-19T12:57:02.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:02.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:57:02.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672976 2026-02-19T12:57:02.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:02.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672976 2026-02-19T12:57:02.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672976' 2026-02-19T12:57:02.849 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 42949672976 2026-02-19T12:57:02.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:57:03.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672976 -lt 42949672976 2026-02-19T12:57:03.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:03.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509455 2026-02-19T12:57:03.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:03.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:57:03.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509455 2026-02-19T12:57:03.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:03.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509455 2026-02-19T12:57:03.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509455' 2026-02-19T12:57:03.018 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 64424509455 2026-02-19T12:57:03.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:57:03.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509456 -lt 64424509455 2026-02-19T12:57:03.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:03.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-292057776133 2026-02-19T12:57:03.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:03.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:57:03.177 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-292057776133 2026-02-19T12:57:03.177 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:03.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776133 2026-02-19T12:57:03.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 292057776133' 2026-02-19T12:57:03.179 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 292057776133 2026-02-19T12:57:03.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:57:03.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776133 -lt 292057776133 2026-02-19T12:57:03.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:03.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182415 2026-02-19T12:57:03.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:03.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:57:03.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-107374182415 2026-02-19T12:57:03.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:03.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182415 2026-02-19T12:57:03.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 107374182415' 2026-02-19T12:57:03.349 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.4 seq 107374182415 2026-02-19T12:57:03.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:57:03.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182415 -lt 107374182415 2026-02-19T12:57:03.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:03.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-124554051597 2026-02-19T12:57:03.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:03.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:57:03.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-124554051597 2026-02-19T12:57:03.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:03.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051597 2026-02-19T12:57:03.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 124554051597' 2026-02-19T12:57:03.514 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.5 seq 124554051597 2026-02-19T12:57:03.514 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:57:03.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051597 -lt 124554051597 2026-02-19T12:57:03.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:03.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-146028888077 2026-02-19T12:57:03.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:03.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:57:03.677 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-146028888077 2026-02-19T12:57:03.678 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:03.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888077 2026-02-19T12:57:03.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 146028888077' 2026-02-19T12:57:03.679 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.6 seq 146028888077 2026-02-19T12:57:03.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:57:03.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888077 -lt 146028888077 2026-02-19T12:57:03.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:03.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724556 2026-02-19T12:57:03.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:03.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:57:03.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-167503724556 2026-02-19T12:57:03.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:03.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724556 2026-02-19T12:57:03.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 167503724556' 2026-02-19T12:57:03.848 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.7 seq 167503724556 2026-02-19T12:57:03.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:57:04.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724556 -lt 167503724556 2026-02-19T12:57:04.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:04.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593739 2026-02-19T12:57:04.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:04.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:57:04.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-184683593739 2026-02-19T12:57:04.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:04.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593739 2026-02-19T12:57:04.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 184683593739' 2026-02-19T12:57:04.009 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.8 seq 184683593739 2026-02-19T12:57:04.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:57:04.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593740 -lt 184683593739 2026-02-19T12:57:04.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:04.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-313532612610 2026-02-19T12:57:04.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:04.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:57:04.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-313532612610 2026-02-19T12:57:04.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:04.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=313532612610 2026-02-19T12:57:04.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 313532612610' 2026-02-19T12:57:04.177 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.9 seq 313532612610 2026-02-19T12:57:04.177 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:57:04.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 313532612610 -lt 313532612610 2026-02-19T12:57:04.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:57:04.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:04.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:04.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:57:04.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:04.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:04.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:04.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:04.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:04.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:04.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:04.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:57:04.705 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:04.705 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:04.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:04.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:57:04.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:57:04.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:57:04.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:57:04.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:304: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:57:04.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:321: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 5 2026-02-19T12:57:04.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:286: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-19T12:57:04.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:287: corrupt_and_repair_one: local poolname=ecpool 2026-02-19T12:57:04.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:288: corrupt_and_repair_one: local osd=5 2026-02-19T12:57:04.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:293: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:57:04.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:57:04.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:57:04.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-19T12:57:04.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:57:04.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:57:04.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:57:04.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:57:04.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-19T12:57:04.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:57:04.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-19T12:57:04.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:57:04.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:57:04.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:57:04.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:57:04.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:57:05.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:57:05.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:57:05.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:57:05.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:57:05.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-19T12:57:05.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:57:05.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:57:05.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-19T12:57:05.418 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-19T12:57:05.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-19T12:57:05.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:57:05.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:57:05.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-19T12:57:05.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:57:05.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:57:05.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:57:05.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:57:05.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:57:05.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:57:05.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-19T12:57:05.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-19T12:57:05.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:57:05.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:57:05.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:57:05.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:57:05.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:57:05.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:57:05.510 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:57:05.510 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:57:05.510 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:57:05.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:05.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:05.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:57:05.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:57:05.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:57:05.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:57:05.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:57:05.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:57:05.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:57:05.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:57:05.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:57:05.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-19T12:57:05.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-19T12:57:05.513 INFO:tasks.workunit.client.0.trial144.stderr:start osd.5 2026-02-19T12:57:05.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:57:05.513 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-19T12:57:05.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-19T12:57:05.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:57:05.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:57:05.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:57:05.529 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:57:05.525+0000 7fceb045ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:05.531 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:57:05.529+0000 7fceb045ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:05.532 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:57:05.529+0000 7fceb045ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:05.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-19T12:57:05.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:57:05.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T12:57:05.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:57:05.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:57:05.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:05.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:57:05.673 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:57:05.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:05.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:57:05.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:57:05.952 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:57:05.949+0000 7fceb045ba40 -1 Falling back to public interface 2026-02-19T12:57:06.551 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:57:06.549+0000 7fceb045ba40 -1 osd.5 75 log_to_monitors true 2026-02-19T12:57:06.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:57:06.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:06.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:57:06.834 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:57:06.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:06.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:57:06.995 INFO:tasks.workunit.client.0.trial144.stderr:osd.5 up in weight 1 up_from 79 up_thru 79 down_at 76 last_clean_interval [29,75) [v2:127.0.0.1:6842/1606445661,v1:127.0.0.1:6843/1606445661] [v2:127.0.0.1:6844/1606445661,v1:127.0.0.1:6845/1606445661] exists,up f2b41730-2834-41c5-a57f-2a8a981fe2c0 2026-02-19T12:57:06.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:57:06.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:57:06.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:57:06.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:57:06.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:57:06.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:57:06.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:57:06.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:57:06.997 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:57:06.997 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:57:06.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:57:06.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:57:06.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:57:07.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:57:07.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:57:07.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:57:07.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:57:07.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:57:07.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:57:07.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:57:07.270 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:57:07.270 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:57:07.270 INFO:tasks.workunit.client.0.trial144.stderr:3 2026-02-19T12:57:07.270 INFO:tasks.workunit.client.0.trial144.stderr:4 2026-02-19T12:57:07.270 INFO:tasks.workunit.client.0.trial144.stderr:5 2026-02-19T12:57:07.270 INFO:tasks.workunit.client.0.trial144.stderr:6 2026-02-19T12:57:07.270 INFO:tasks.workunit.client.0.trial144.stderr:7 2026-02-19T12:57:07.270 INFO:tasks.workunit.client.0.trial144.stderr:8 2026-02-19T12:57:07.270 INFO:tasks.workunit.client.0.trial144.stderr:9' 2026-02-19T12:57:07.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:57:07.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:07.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:57:07.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836499 2026-02-19T12:57:07.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836499 2026-02-19T12:57:07.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499' 2026-02-19T12:57:07.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:07.342 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:57:07.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672978 2026-02-19T12:57:07.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672978 2026-02-19T12:57:07.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672978' 2026-02-19T12:57:07.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:07.417 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:57:07.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509458 2026-02-19T12:57:07.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509458 2026-02-19T12:57:07.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-64424509458' 2026-02-19T12:57:07.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:07.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:57:07.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776136 2026-02-19T12:57:07.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776136 2026-02-19T12:57:07.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-64424509458 3-292057776136' 2026-02-19T12:57:07.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:07.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:57:07.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182417 2026-02-19T12:57:07.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182417 2026-02-19T12:57:07.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-64424509458 3-292057776136 4-107374182417' 2026-02-19T12:57:07.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:07.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:57:07.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416386 2026-02-19T12:57:07.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416386 2026-02-19T12:57:07.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-64424509458 3-292057776136 4-107374182417 5-339302416386' 2026-02-19T12:57:07.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:07.713 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:57:07.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888079 2026-02-19T12:57:07.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888079 2026-02-19T12:57:07.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-64424509458 3-292057776136 4-107374182417 5-339302416386 6-146028888079' 2026-02-19T12:57:07.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:07.788 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:57:07.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724558 2026-02-19T12:57:07.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724558 2026-02-19T12:57:07.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-64424509458 3-292057776136 4-107374182417 5-339302416386 6-146028888079 7-167503724558' 2026-02-19T12:57:07.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:07.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:57:07.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593741 2026-02-19T12:57:07.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593741 2026-02-19T12:57:07.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-64424509458 3-292057776136 4-107374182417 5-339302416386 6-146028888079 7-167503724558 8-184683593741' 2026-02-19T12:57:07.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:07.930 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:57:08.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=313532612612 2026-02-19T12:57:08.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 313532612612 2026-02-19T12:57:08.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-64424509458 3-292057776136 4-107374182417 5-339302416386 6-146028888079 7-167503724558 8-184683593741 9-313532612612' 2026-02-19T12:57:08.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:08.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836499 2026-02-19T12:57:08.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:08.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:57:08.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836499 2026-02-19T12:57:08.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:08.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836499 2026-02-19T12:57:08.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2026-02-19T12:57:08.006 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836499 2026-02-19T12:57:08.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:08.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836499 -lt 21474836499 2026-02-19T12:57:08.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:08.177 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672978 2026-02-19T12:57:08.177 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:08.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:57:08.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672978 2026-02-19T12:57:08.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:08.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672978 2026-02-19T12:57:08.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672978' 2026-02-19T12:57:08.180 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 42949672978 2026-02-19T12:57:08.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:57:08.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672979 -lt 42949672978 2026-02-19T12:57:08.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:08.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509458 2026-02-19T12:57:08.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:08.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:57:08.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509458 2026-02-19T12:57:08.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:08.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509458 2026-02-19T12:57:08.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509458' 2026-02-19T12:57:08.351 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 64424509458 2026-02-19T12:57:08.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:57:08.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509458 -lt 64424509458 2026-02-19T12:57:08.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:08.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-292057776136 2026-02-19T12:57:08.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:08.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:57:08.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-292057776136 2026-02-19T12:57:08.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:08.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776136 2026-02-19T12:57:08.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 292057776136' 2026-02-19T12:57:08.520 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 292057776136 2026-02-19T12:57:08.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:57:08.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776136 -lt 292057776136 2026-02-19T12:57:08.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:08.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182417 2026-02-19T12:57:08.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:08.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:57:08.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-107374182417 2026-02-19T12:57:08.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:08.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182417 2026-02-19T12:57:08.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 107374182417' 2026-02-19T12:57:08.696 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.4 seq 107374182417 2026-02-19T12:57:08.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:57:08.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182417 -lt 107374182417 2026-02-19T12:57:08.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:08.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-339302416386 2026-02-19T12:57:08.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:08.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:57:08.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-339302416386 2026-02-19T12:57:08.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:08.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=339302416386 2026-02-19T12:57:08.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 339302416386' 2026-02-19T12:57:08.859 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.5 seq 339302416386 2026-02-19T12:57:08.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:57:09.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 339302416386 -lt 339302416386 2026-02-19T12:57:09.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:09.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-146028888079 2026-02-19T12:57:09.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:09.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:57:09.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-146028888079 2026-02-19T12:57:09.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:09.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888079 2026-02-19T12:57:09.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 146028888079' 2026-02-19T12:57:09.025 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.6 seq 146028888079 2026-02-19T12:57:09.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:57:09.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888079 -lt 146028888079 2026-02-19T12:57:09.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:09.194 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724558 2026-02-19T12:57:09.194 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:09.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:57:09.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-167503724558 2026-02-19T12:57:09.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:09.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724558 2026-02-19T12:57:09.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 167503724558' 2026-02-19T12:57:09.198 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.7 seq 167503724558 2026-02-19T12:57:09.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:57:09.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724558 -lt 167503724558 2026-02-19T12:57:09.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:09.363 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593741 2026-02-19T12:57:09.364 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:09.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:57:09.366 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-184683593741 2026-02-19T12:57:09.366 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:09.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593741 2026-02-19T12:57:09.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 184683593741' 2026-02-19T12:57:09.368 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.8 seq 184683593741 2026-02-19T12:57:09.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:57:09.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593741 -lt 184683593741 2026-02-19T12:57:09.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:09.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-313532612612 2026-02-19T12:57:09.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:09.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:57:09.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-313532612612 2026-02-19T12:57:09.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:09.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=313532612612 2026-02-19T12:57:09.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 313532612612' 2026-02-19T12:57:09.537 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.9 seq 313532612612 2026-02-19T12:57:09.537 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:57:09.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 313532612612 -lt 313532612612 2026-02-19T12:57:09.705 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:57:09.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:09.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:09.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:57:09.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:09.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:09.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:09.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:09.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:09.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:09.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:10.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:57:10.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:10.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:10.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:10.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:57:10.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:57:10.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:57:10.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-19T12:57:10.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:57:10.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:57:10.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:57:10.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:57:10.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: local pg=2.0 2026-02-19T12:57:10.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: corrupt_and_repair_one: repair 2.0 2026-02-19T12:57:10.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:57:10.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:57:10.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:10.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:10.420 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:10.420 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:10.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:56:55.450552+0000 2026-02-19T12:57:10.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:57:10.727 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:57:10.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:56:55.450552+0000 2026-02-19T12:57:10.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:57:10.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:56:55.450552+0000 2026-02-19T12:57:10.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:57:10.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:57:10.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:10.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:10.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:10.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:10.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:10.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:10.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:56:55.450552+0000 '>' 2026-02-19T12:56:55.450552+0000 2026-02-19T12:57:10.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:57:11.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:57:11.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:11.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:11.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:11.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:11.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:11.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:12.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:56:55.450552+0000 '>' 2026-02-19T12:56:55.450552+0000 2026-02-19T12:57:12.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:57:13.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:57:13.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:13.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:13.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:13.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:13.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:13.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:13.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:56:55.450552+0000 '>' 2026-02-19T12:56:55.450552+0000 2026-02-19T12:57:13.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:57:14.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:57:14.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:14.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:14.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:14.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:14.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:14.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:14.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:57:11.513763+0000 '>' 2026-02-19T12:56:55.450552+0000 2026-02-19T12:57:14.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:57:14.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:302: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:57:14.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:57:14.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:57:14.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-19T12:57:14.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:57:14.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:57:14.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:57:14.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:57:14.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-19T12:57:14.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:57:14.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-19T12:57:14.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:57:14.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:57:14.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:57:14.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:57:14.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:57:14.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:57:14.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:57:14.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:57:14.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:57:14.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-19T12:57:14.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:57:14.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:57:14.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-19T12:57:14.760 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:57:14.761 INFO:tasks.workunit.client.0.trial144.stdout:hinfo_key 2026-02-19T12:57:14.761 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:57:14.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-19T12:57:14.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:57:14.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:57:14.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-19T12:57:14.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:57:14.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:57:14.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:57:14.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:57:14.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:57:14.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:57:14.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-19T12:57:14.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-19T12:57:14.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:57:14.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:57:14.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:57:14.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:57:14.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:57:14.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:57:14.853 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:57:14.853 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:57:14.854 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:57:14.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:14.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:14.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:57:14.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:57:14.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:57:14.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:57:14.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:57:14.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:57:14.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:57:14.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:57:14.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:57:14.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-19T12:57:14.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-19T12:57:14.857 INFO:tasks.workunit.client.0.trial144.stderr:start osd.9 2026-02-19T12:57:14.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:57:14.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-19T12:57:14.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-19T12:57:14.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:57:14.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:57:14.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:57:14.873 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:57:14.868+0000 7f8f23b57a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:14.874 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:57:14.872+0000 7f8f23b57a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:14.875 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:57:14.872+0000 7f8f23b57a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:15.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-19T12:57:15.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:57:15.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-19T12:57:15.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:57:15.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:57:15.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:15.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:57:15.020 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:57:15.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:15.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:57:15.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:57:15.296 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:57:15.292+0000 7f8f23b57a40 -1 Falling back to public interface 2026-02-19T12:57:15.914 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:57:15.908+0000 7f8f23b57a40 -1 osd.9 81 log_to_monitors true 2026-02-19T12:57:16.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:57:16.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:16.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:57:16.178 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:57:16.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:16.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:57:16.332 INFO:tasks.workunit.client.0.trial144.stderr:osd.9 up in weight 1 up_from 85 up_thru 0 down_at 82 last_clean_interval [73,81) [v2:127.0.0.1:6874/4222335531,v1:127.0.0.1:6875/4222335531] [v2:127.0.0.1:6876/4222335531,v1:127.0.0.1:6877/4222335531] exists,up 9f6223c2-19a3-4d20-afdc-51e479427486 2026-02-19T12:57:16.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:57:16.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:57:16.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:57:16.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:57:16.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:57:16.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:57:16.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:57:16.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:57:16.333 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:57:16.333 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:57:16.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:57:16.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:57:16.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:57:16.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:57:16.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:57:16.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:57:16.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:57:16.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:57:16.448 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:57:16.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:57:16.603 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:57:16.603 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T12:57:16.603 INFO:tasks.workunit.client.0.trial144.stderr:3 2026-02-19T12:57:16.603 INFO:tasks.workunit.client.0.trial144.stderr:4 2026-02-19T12:57:16.603 INFO:tasks.workunit.client.0.trial144.stderr:5 2026-02-19T12:57:16.603 INFO:tasks.workunit.client.0.trial144.stderr:6 2026-02-19T12:57:16.603 INFO:tasks.workunit.client.0.trial144.stderr:7 2026-02-19T12:57:16.603 INFO:tasks.workunit.client.0.trial144.stderr:8 2026-02-19T12:57:16.603 INFO:tasks.workunit.client.0.trial144.stderr:9' 2026-02-19T12:57:16.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:57:16.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:16.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:57:16.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836502 2026-02-19T12:57:16.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836502 2026-02-19T12:57:16.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502' 2026-02-19T12:57:16.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:16.679 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:57:16.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672981 2026-02-19T12:57:16.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672981 2026-02-19T12:57:16.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-42949672981' 2026-02-19T12:57:16.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:16.752 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:57:16.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509461 2026-02-19T12:57:16.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509461 2026-02-19T12:57:16.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-42949672981 2-64424509461' 2026-02-19T12:57:16.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:16.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:57:16.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776138 2026-02-19T12:57:16.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776138 2026-02-19T12:57:16.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-42949672981 2-64424509461 3-292057776138' 2026-02-19T12:57:16.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:16.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:57:16.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182420 2026-02-19T12:57:16.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182420 2026-02-19T12:57:16.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-42949672981 2-64424509461 3-292057776138 4-107374182420' 2026-02-19T12:57:16.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:16.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:57:17.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416389 2026-02-19T12:57:17.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416389 2026-02-19T12:57:17.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-42949672981 2-64424509461 3-292057776138 4-107374182420 5-339302416389' 2026-02-19T12:57:17.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:17.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:57:17.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888082 2026-02-19T12:57:17.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888082 2026-02-19T12:57:17.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-42949672981 2-64424509461 3-292057776138 4-107374182420 5-339302416389 6-146028888082' 2026-02-19T12:57:17.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:17.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:57:17.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724561 2026-02-19T12:57:17.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724561 2026-02-19T12:57:17.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-42949672981 2-64424509461 3-292057776138 4-107374182420 5-339302416389 6-146028888082 7-167503724561' 2026-02-19T12:57:17.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:17.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:57:17.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593744 2026-02-19T12:57:17.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593744 2026-02-19T12:57:17.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-42949672981 2-64424509461 3-292057776138 4-107374182420 5-339302416389 6-146028888082 7-167503724561 8-184683593744' 2026-02-19T12:57:17.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:17.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:57:17.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=365072220162 2026-02-19T12:57:17.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 365072220162 2026-02-19T12:57:17.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836502 1-42949672981 2-64424509461 3-292057776138 4-107374182420 5-339302416389 6-146028888082 7-167503724561 8-184683593744 9-365072220162' 2026-02-19T12:57:17.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:17.331 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836502 2026-02-19T12:57:17.331 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:17.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:57:17.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836502 2026-02-19T12:57:17.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:17.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836502 2026-02-19T12:57:17.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836502' 2026-02-19T12:57:17.334 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836502 2026-02-19T12:57:17.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:17.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836501 -lt 21474836502 2026-02-19T12:57:17.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:57:18.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:57:18.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:18.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836502 -lt 21474836502 2026-02-19T12:57:18.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:18.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672981 2026-02-19T12:57:18.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:18.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:57:18.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672981 2026-02-19T12:57:18.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:18.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672981 2026-02-19T12:57:18.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672981' 2026-02-19T12:57:18.664 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 42949672981 2026-02-19T12:57:18.665 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:57:18.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672982 -lt 42949672981 2026-02-19T12:57:18.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:18.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509461 2026-02-19T12:57:18.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:18.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:57:18.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509461 2026-02-19T12:57:18.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:18.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509461 2026-02-19T12:57:18.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509461' 2026-02-19T12:57:18.825 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 64424509461 2026-02-19T12:57:18.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:57:18.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509461 -lt 64424509461 2026-02-19T12:57:18.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:18.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-292057776138 2026-02-19T12:57:18.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:18.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:57:18.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-292057776138 2026-02-19T12:57:18.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:18.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776138 2026-02-19T12:57:18.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 292057776138' 2026-02-19T12:57:18.990 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.3 seq 292057776138 2026-02-19T12:57:18.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:57:19.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776139 -lt 292057776138 2026-02-19T12:57:19.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:19.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182420 2026-02-19T12:57:19.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:19.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:57:19.163 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-107374182420 2026-02-19T12:57:19.163 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:19.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182420 2026-02-19T12:57:19.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 107374182420' 2026-02-19T12:57:19.164 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.4 seq 107374182420 2026-02-19T12:57:19.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:57:19.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182420 -lt 107374182420 2026-02-19T12:57:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:19.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-339302416389 2026-02-19T12:57:19.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:57:19.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-339302416389 2026-02-19T12:57:19.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:19.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=339302416389 2026-02-19T12:57:19.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 339302416389' 2026-02-19T12:57:19.334 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.5 seq 339302416389 2026-02-19T12:57:19.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:57:19.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 339302416389 -lt 339302416389 2026-02-19T12:57:19.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:19.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-146028888082 2026-02-19T12:57:19.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:19.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:57:19.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-146028888082 2026-02-19T12:57:19.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:19.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888082 2026-02-19T12:57:19.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 146028888082' 2026-02-19T12:57:19.521 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.6 seq 146028888082 2026-02-19T12:57:19.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:57:19.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888082 -lt 146028888082 2026-02-19T12:57:19.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:19.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724561 2026-02-19T12:57:19.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:19.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:57:19.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-167503724561 2026-02-19T12:57:19.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:19.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724561 2026-02-19T12:57:19.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 167503724561' 2026-02-19T12:57:19.670 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.7 seq 167503724561 2026-02-19T12:57:19.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:57:19.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724561 -lt 167503724561 2026-02-19T12:57:19.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:19.841 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593744 2026-02-19T12:57:19.841 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:19.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:57:19.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-184683593744 2026-02-19T12:57:19.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:19.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593744 2026-02-19T12:57:19.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 184683593744' 2026-02-19T12:57:19.845 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.8 seq 184683593744 2026-02-19T12:57:19.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:57:20.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593744 -lt 184683593744 2026-02-19T12:57:20.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:20.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-365072220162 2026-02-19T12:57:20.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:20.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:57:20.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-365072220162 2026-02-19T12:57:20.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:20.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=365072220162 2026-02-19T12:57:20.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 365072220162' 2026-02-19T12:57:20.019 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.9 seq 365072220162 2026-02-19T12:57:20.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:57:20.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 365072220162 -lt 365072220162 2026-02-19T12:57:20.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:57:20.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:20.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:20.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:57:20.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:20.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:20.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:20.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:20.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:20.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:20.376 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:20.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:57:20.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:20.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:20.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:20.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:57:20.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:57:20.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:57:20.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:57:20.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:304: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:57:20.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:322: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 5 9 2026-02-19T12:57:20.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:245: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-19T12:57:20.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:246: corrupt_and_repair_two: local poolname=ecpool 2026-02-19T12:57:20.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:247: corrupt_and_repair_two: local first=5 2026-02-19T12:57:20.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_two: local second=9 2026-02-19T12:57:20.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_two: pids= 2026-02-19T12:57:20.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:57:20.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:57:20.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:57:20.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 220574"' 2026-02-19T12:57:20.755 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 220574' 2026-02-19T12:57:20.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 9 SOMETHING remove 2026-02-19T12:57:20.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:57:20.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:57:20.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 220576"' 2026-02-19T12:57:20.755 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 220576' 2026-02-19T12:57:20.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:57:20.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_two: wait_background pids 2026-02-19T12:57:20.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 220574 220576' 2026-02-19T12:57:20.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:57:20.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:57:20.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 220574 2026-02-19T12:57:20.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/220577: /' 2026-02-19T12:57:20.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 9 SOMETHING remove 2026-02-19T12:57:20.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/220579: /' 2026-02-19T12:57:21.355 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:57:21.355 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:57:21.355 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-19T12:57:21.355 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:57:21.355 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING remove 2026-02-19T12:57:21.355 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:57:21.355 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:57:21.355 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-19T12:57:21.355 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:57:21.356 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-19T12:57:21.356 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:57:21.356 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:57:21.356 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:57:21.356 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:57:21.356 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:57:21.356 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:57:21.356 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING remove 2026-02-19T12:57:21.356 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:57:21.356 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:57:21.356 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-19T12:57:21.356 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:57:21.357 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:57:21.357 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING remove 2026-02-19T12:57:21.357 INFO:tasks.workunit.client.0.trial144.stderr:220579: remove 2#2:eb822e21:::SOMETHING:head# 2026-02-19T12:57:21.357 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-19T12:57:21.357 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:57:21.357 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:57:21.357 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-19T12:57:21.357 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:57:21.357 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:57:21.357 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:57:21.357 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:57:21.358 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:57:21.358 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:57:21.359 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-19T12:57:21.359 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-19T12:57:21.359 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:57:21.359 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:57:21.359 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:57:21.359 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:57:21.359 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:57:21.359 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:57:21.359 INFO:tasks.workunit.client.0.trial144.stderr:220579: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:57:21.359 INFO:tasks.workunit.client.0.trial144.stderr:220579: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:57:21.359 INFO:tasks.workunit.client.0.trial144.stderr:220579: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:57:21.359 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:21.360 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:21.360 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:57:21.360 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:57:21.360 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:57:21.360 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:57:21.360 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:57:21.360 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:57:21.360 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:57:21.360 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:57:21.360 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:57:21.360 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-19T12:57:21.360 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-19T12:57:21.361 INFO:tasks.workunit.client.0.trial144.stderr:220579: start osd.9 2026-02-19T12:57:21.361 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:57:21.371 INFO:tasks.workunit.client.0.trial144.stderr:2220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:57:21.371 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:57:21.371 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-19T12:57:21.371 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:57:21.371 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:57:21.371 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:57:21.372 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:57:21.372 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-19T12:57:21.372 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:57:21.372 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-19T12:57:21.372 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:57:21.372 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:57:21.372 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:57:21.372 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:57:21.372 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:57:21.372 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:57:21.372 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:57:21.373 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:57:21.373 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:57:21.373 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-19T12:57:21.373 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:57:21.373 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:57:21.373 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-19T12:57:21.373 INFO:tasks.workunit.client.0.trial144.stderr:220577: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-19T12:57:21.373 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-19T12:57:21.373 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:57:21.373 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:57:21.374 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-19T12:57:21.374 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:57:21.374 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:57:21.374 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:57:21.374 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:57:21.375 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:57:21.375 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:57:21.375 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-19T12:57:21.375 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-19T12:57:21.375 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:57:21.376 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:57:21.376 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:57:21.376 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:57:21.376 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:57:21.376 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:57:21.376 INFO:tasks.workunit.client.0.trial144.stderr:220577: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:57:21.376 INFO:tasks.workunit.client.0.trial144.stderr:220577: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:57:21.376 INFO:tasks.workunit.client.0.trial144.stderr:220577: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:57:21.376 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:21.376 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:21.376 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:57:21.377 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:57:21.377 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:57:21.377 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:57:21.377 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:57:21.377 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:57:21.377 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:57:21.377 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:57:21.377 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:57:21.377 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-19T12:57:21.377 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-19T12:57:21.377 INFO:tasks.workunit.client.0.trial144.stderr:220577: start osd.5 2026-02-19T12:57:21.378 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:57:22.842 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-19T12:57:22.842 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-19T12:57:22.843 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:57:22.843 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:57:22.843 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:57:22.843 INFO:tasks.workunit.client.0.trial144.stderr:220579: 2026-02-19T12:57:21.376+0000 7f98fbed2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:22.843 INFO:tasks.workunit.client.0.trial144.stderr:220579: 2026-02-19T12:57:21.376+0000 7f98fbed2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:22.843 INFO:tasks.workunit.client.0.trial144.stderr:220579: 2026-02-19T12:57:21.380+0000 7f98fbed2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:22.843 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-19T12:57:22.843 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:57:22.843 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-19T12:57:22.843 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:57:22.843 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:57:22.844 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:22.844 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:57:22.844 INFO:tasks.workunit.client.0.trial144.stderr:220579: 0 2026-02-19T12:57:22.844 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:22.844 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:57:22.844 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:57:22.844 INFO:tasks.workunit.client.0.trial144.stderr:220579: 2026-02-19T12:57:21.832+0000 7f98fbed2a40 -1 Falling back to public interface 2026-02-19T12:57:22.844 INFO:tasks.workunit.client.0.trial144.stderr:220579: 2026-02-19T12:57:21.832+0000 7f98fae92640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6842/0: (98) Address already in use 2026-02-19T12:57:22.844 INFO:tasks.workunit.client.0.trial144.stderr:220579: 2026-02-19T12:57:22.496+0000 7f98fbed2a40 -1 osd.9 86 log_to_monitors true 2026-02-19T12:57:22.844 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:57:22.844 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:22.844 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:57:22.845 INFO:tasks.workunit.client.0.trial144.stderr:220579: 1 2026-02-19T12:57:22.845 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:22.845 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:57:22.845 INFO:tasks.workunit.client.0.trial144.stderr:220579: osd.9 up in weight 1 up_from 90 up_thru 0 down_at 87 last_clean_interval [85,86) [v2:127.0.0.1:6843/2813089807,v1:127.0.0.1:6845/2813089807] [v2:127.0.0.1:6847/2813089807,v1:127.0.0.1:6849/2813089807] exists,up 9f6223c2-19a3-4d20-afdc-51e479427486 2026-02-19T12:57:22.845 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:57:22.845 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:57:22.845 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:57:22.845 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:57:22.845 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:57:22.845 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:57:22.845 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:57:22.845 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:57:22.855 INFO:tasks.workunit.client.0.trial144.stderr:220579: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-help20577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-19T12:57:22.855 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-19T12:57:22.855 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:57:22.855 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:57:22.855 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:57:22.856 INFO:tasks.workunit.client.0.trial144.stderr:220577: 2026-02-19T12:57:21.388+0000 7f882e078a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:22.856 INFO:tasks.workunit.client.0.trial144.stderr:220577: 2026-02-19T12:57:21.388+0000 7f882e078a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:22.856 INFO:tasks.workunit.client.0.trial144.stderr:220577: 2026-02-19T12:57:21.388+0000 7f882e078a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:22.856 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-19T12:57:22.856 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:57:22.856 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T12:57:22.856 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:57:22.856 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:57:22.856 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:22.856 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:57:22.856 INFO:tasks.workunit.client.0.trial144.stderr:220577: 0 2026-02-19T12:57:22.856 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:22.856 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:57:22.857 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:57:22.857 INFO:tasks.workunit.client.0.trial144.stderr:220577: 2026-02-19T12:57:21.832+0000 7f882e078a40 -1 Falling back to public interface 2026-02-19T12:57:22.857 INFO:tasks.workunit.client.0.trial144.stderr:220577: 2026-02-19T12:57:22.512+0000 7f882e078a40 -1 osd.5 86 log_to_monitors true 2026-02-19T12:57:22.857 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:57:22.857 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:22.857 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:57:22.857 INFO:tasks.workunit.client.0.trial144.stderr:220577: 1 2026-02-19T12:57:22.857 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:22.857 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:57:22.857 INFO:tasks.workunit.client.0.trial144.stderr:220577: osd.5 up in weight 1 up_from 90 up_thru 90 down_at 87 last_clean_interval [79,86) [v2:127.0.0.1:6842/3923310958,v1:127.0.0.1:6844/3923310958] [v2:127.0.0.1:6846/3923310958,v1:127.0.0.1:6848/3923310958] exists,up f2b41730-2834-41c5-a57f-2a8a981fe2c0 2026-02-19T12:57:22.857 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:57:22.857 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:57:22.858 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:57:22.858 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:57:22.858 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:57:22.858 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:57:22.858 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:57:22.858 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:57:22.858 INFO:tasks.workunit.client.0.trial144.stderr:220577: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:57:23.410 INFO:tasks.workunit.client.0.trial144.stderr:220577: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: gers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:57:23.410 INFO:tasks.workunit.client.0.trial144.stderr:220579: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:57:23.410 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:57:23.410 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:57:23.410 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:57:23.410 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:57:23.410 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:57:23.411 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:57:23.411 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:57:23.411 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:57:23.411 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:57:23.411 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:57:23.411 INFO:tasks.workunit.client.0.trial144.stderr:220579: 1 2026-02-19T12:57:23.411 INFO:tasks.workunit.client.0.trial144.stderr:220579: 2 2026-02-19T12:57:23.411 INFO:tasks.workunit.client.0.trial144.stderr:220579: 3 2026-02-19T12:57:23.411 INFO:tasks.workunit.client.0.trial144.stderr:220579: 4 2026-02-19T12:57:23.411 INFO:tasks.workunit.client.0.trial144.stderr:220579: 5 2026-02-19T12:57:23.411 INFO:tasks.workunit.client.0.trial144.stderr:220579: 6 2026-02-19T12:57:23.411 INFO:tasks.workunit.client.0.trial144.stderr:220579: 7 2026-02-19T12:57:23.412 INFO:tasks.workunit.client.0.trial144.stderr:220579: 8 2026-02-19T12:57:23.412 INFO:tasks.workunit.client.0.trial144.stderr:220579: 9' 2026-02-19T12:57:23.412 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:57:23.412 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.412 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:57:23.412 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836504 2026-02-19T12:57:23.412 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836504 2026-02-19T12:57:23.412 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504' 2026-02-19T12:57:23.412 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.412 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:57:23.412 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672985 2026-02-19T12:57:23.412 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672985 2026-02-19T12:57:23.413 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-42949672985' 2026-02-19T12:57:23.413 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.413 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:57:23.413 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509464 2026-02-19T12:57:23.413 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509464 2026-02-19T12:57:23.413 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-42949672985 2-64424509464' 2026-02-19T12:57:23.413 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.413 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:57:23.413 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776142 2026-02-19T12:57:23.413 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776142 2026-02-19T12:57:23.413 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-42949672985 2-64424509464 3-292057776142' 2026-02-19T12:57:23.413 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standaloneet_timeout_delays: echo true 2026-02-19T12:57:23.414 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:57:23.414 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:57:23.414 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:57:23.414 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:57:23.414 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:57:23.414 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:57:23.414 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:57:23.414 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:57:23.414 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:57:23.414 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:57:23.415 INFO:tasks.workunit.client.0.trial144.stderr:220577: 1 2026-02-19T12:57:23.415 INFO:tasks.workunit.client.0.trial144.stderr:220577: 2 2026-02-19T12:57:23.415 INFO:tasks.workunit.client.0.trial144.stderr:220577: 3 2026-02-19T12:57:23.415 INFO:tasks.workunit.client.0.trial144.stderr:220577: 4 2026-02-19T12:57:23.415 INFO:tasks.workunit.client.0.trial144.stderr:220577: 5 2026-02-19T12:57:23.415 INFO:tasks.workunit.client.0.trial144.stderr:220577: 6 2026-02-19T12:57:23.415 INFO:tasks.workunit.client.0.trial144.stderr:220577: 7 2026-02-19T12:57:23.415 INFO:tasks.workunit.client.0.trial144.stderr:220577: 8 2026-02-19T12:57:23.415 INFO:tasks.workunit.client.0.trial144.stderr:220577: 9' 2026-02-19T12:57:23.415 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:57:23.415 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.416 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:57:23.416 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836505 2026-02-19T12:57:23.416 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836505 2026-02-19T12:57:23.416 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505' 2026-02-19T12:57:23.416 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.416 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:57:23.416 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672984 2026-02-19T12:57:23.416 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672984 2026-02-19T12:57:23.416 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-42949672984' 2026-02-19T12:57:23.416 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.416 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:57:23.417 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509463 2026-02-19T12:57:23.417 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509463 2026-02-19T12:57:23.417 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-42949672984 2-64424509463' 2026-02-19T12:57:23.417 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.417 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:57:23.417 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776141 2026-02-19T12:57:23.417 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776141 2026-02-19T12:57:23.417 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-42949672984 2-64424509463 3-292057776141' 2026-02-19T12:57:23.417 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.845 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:57:23.845 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182422 2026-02-19T12:57:23.845 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182422 2026-02-19T12:57:23.845 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-42949672984 2-64424509463 3-292057776141 4-107374182422' 2026-02-19T12:57:23.845 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.845 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:57:23.845 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=386547056642 2026-02-19T12:57:23.845 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 386547056642 2026-02-19T12:57:23.845 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-42949672984 2-64424509463 3-292057776141 4-107374182422 5-386547056642' 2026-02-19T12:57:23.846 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.846 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:57:23.846 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888084 2026-02-19T12:57:23.846 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888084 2026-02-19T12:57:23.846 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-42949672984 2-64424509463 3-292057776141 4-107374182422 5-386547056642 6-146028888084' 2026-02-19T12:57:23.846 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.846 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:57:23.846 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724563 2026-02-19T12:57:23.846 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724563 2026-02-19T12:57:23.846 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-42949672984 2-64424509463 3-292057776141 4-107374182422 5-386547056642 6-146028888084 7-167503724563' 2026-02-19T12:57:23.847 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.847 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:57:23.847 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593747 2026-02-19T12:57:23.847 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593747 2026-02-19T12:57:23.847 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-42949672984 2-64424509463 3-292057776141 4-107374182422 5-386547056642 6-146028888084 7-167503724563 8-184683593747' 2026-02-19T12:57:23.847 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.847 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:57:23.847 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=386547056642 2026-02-19T12:57:23.847 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 386547056642 2026-02-19T12:57:23.847 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-42949672984 2-64424509463 3-292057776141 4-107374182422 5-386547056642 6-146028888084 7-167503724563 8-184683593747 9-386547056642' 2026-02-19T12:57:23.847 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:23.848 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ub/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.848 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:57:23.848 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182423 2026-02-19T12:57:23.848 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182423 2026-02-19T12:57:23.849 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-42949672985 2-64424509464 3-292057776142 4-107374182423' 2026-02-19T12:57:23.849 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.849 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:57:23.849 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=386547056643 2026-02-19T12:57:23.849 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 386547056643 2026-02-19T12:57:23.849 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-42949672985 2-64424509464 3-292057776142 4-107374182423 5-386547056643' 2026-02-19T12:57:23.849 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.849 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:57:23.849 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888085 2026-02-19T12:57:23.849 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888085 2026-02-19T12:57:23.850 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-42949672985 2-64424509464 3-292057776142 4-107374182423 5-386547056643 6-146028888085' 2026-02-19T12:57:23.850 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.850 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:57:23.850 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724564 2026-02-19T12:57:23.850 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724564 2026-02-19T12:57:23.850 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-42949672985 2-64424509464 3-292057776142 4-107374182423 5-386547056643 6-146028888085 7-167503724564' 2026-02-19T12:57:23.850 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.850 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:57:23.850 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593748 2026-02-19T12:57:23.850 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593748 2026-02-19T12:57:23.851 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-42949672985 2-64424509464 3-292057776142 4-107374182423 5-386547056643 6-146028888085 7-167503724564 8-184683593748' 2026-02-19T12:57:23.851 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:23.851 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:57:23.851 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=386547056643 2026-02-19T12:57:23.851 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 386547056643 2026-02-19T12:57:25.501 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836504 1-42949672985 2-64424509464 3-292057776142 4-107374182423 5-386547056643 6-146028888085 7-167503724564 8-184683593748 9-386547056643' 2026-02-19T12:57:25.501 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:25.501 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836504 2026-02-19T12:57:25.501 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:25.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:57:25.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836504 2026-02-19T12:57:25.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:25.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836504 2026-02-19T12:57:25.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836504' 2026-02-19T12:57:25.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: waiting osd.0 seq 21474836504 2026-02-19T12:57:25.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:25.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836503 -lt 21474836504 2026-02-19T12:57:25.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:57:25.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:57:25.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:25.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836506 -lt 21474836504 2026-02-19T12:57:25.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:25.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672985 2026-02-19T12:57:25.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:25.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:57:25.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672985 2026-02-19T12:57:25.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:25.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672985 2026-02-19T12:57:25.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672985' 2026-02-19T12:57:25.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: waiting osd.1 seq 42949672985 2026-02-19T12:57:25.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:57:25.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672985 -lt 42949672985 2026-02-19T12:57:25.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:25.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509464 2026-02-19T12:57:25.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:25.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:57:25.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509464 2026-02-19T12:57:25.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:25.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509464 2026-02-19T12:57:25.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509464' 2026-02-19T12:57:25.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: waiting osd.2 seq 64424509464 2026-02-19T12:57:25.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:57:25.510 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270:untu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836505 2026-02-19T12:57:25.510 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:25.510 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:57:25.510 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836505 2026-02-19T12:57:25.510 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:25.510 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836505 2026-02-19T12:57:25.511 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836505' 2026-02-19T12:57:25.511 INFO:tasks.workunit.client.0.trial144.stderr:220577: waiting osd.0 seq 21474836505 2026-02-19T12:57:25.511 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:25.511 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836503 -lt 21474836505 2026-02-19T12:57:25.511 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:57:25.511 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:57:25.511 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:25.511 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836506 -lt 21474836505 2026-02-19T12:57:25.511 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:25.511 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672984 2026-02-19T12:57:25.511 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:25.512 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:57:25.512 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672984 2026-02-19T12:57:25.512 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:25.512 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672984 2026-02-19T12:57:25.512 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672984' 2026-02-19T12:57:25.512 INFO:tasks.workunit.client.0.trial144.stderr:220577: waiting osd.1 seq 42949672984 2026-02-19T12:57:25.512 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:57:25.512 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672985 -lt 42949672984 2026-02-19T12:57:25.512 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:25.512 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509463 2026-02-19T12:57:25.513 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:25.513 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:57:25.513 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509463 2026-02-19T12:57:25.513 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:25.513 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509463 2026-02-19T12:57:25.513 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509463' 2026-02-19T12:57:25.513 INFO:tasks.workunit.client.0.trial144.stderr:220577: waiting osd.2 seq 64424509463 2026-02-19T12:57:25.513 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:57:25.513 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509464 -lt 64424509463 2026-02-19T12:57:26.002 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush flush_pg_stats: test 64424509464 -lt 64424509464 2026-02-19T12:57:26.003 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:26.003 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-292057776142 2026-02-19T12:57:26.003 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:26.003 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:57:26.003 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-292057776142 2026-02-19T12:57:26.003 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:26.003 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776142 2026-02-19T12:57:26.003 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 292057776142' 2026-02-19T12:57:26.003 INFO:tasks.workunit.client.0.trial144.stderr:220579: waiting osd.3 seq 292057776142 2026-02-19T12:57:26.003 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:57:26.004 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776142 -lt 292057776142 2026-02-19T12:57:26.004 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:26.004 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182423 2026-02-19T12:57:26.004 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:26.004 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:57:26.004 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-107374182423 2026-02-19T12:57:26.004 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:26.004 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182423 2026-02-19T12:57:26.004 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 107374182423' 2026-02-19T12:57:26.004 INFO:tasks.workunit.client.0.trial144.stderr:220579: waiting osd.4 seq 107374182423 2026-02-19T12:57:26.004 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:57:26.005 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182424 -lt 107374182423 2026-02-19T12:57:26.005 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:26.005 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-386547056643 2026-02-19T12:57:26.005 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:26.005 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:57:26.005 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-386547056643 2026-02-19T12:57:26.005 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:26.005 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=386547056643 2026-02-19T12:57:26.005 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 386547056643' 2026-02-19T12:57:26.005 INFO:tasks.workunit.client.0.trial144.stderr:220579: waiting osd.5 seq 386547056643 2026-02-19T12:57:26.005 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:57:26.005 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 386547056643 -lt 386547056643 2026-02-19T12:57:26.006 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:26.006 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-146028888085 2026-02-19T12:57:26.006 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:26.014 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ub_pg_stats: for s in $seqs 2026-02-19T12:57:26.014 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-292057776141 2026-02-19T12:57:26.014 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:26.015 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:57:26.015 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-292057776141 2026-02-19T12:57:26.015 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:26.015 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776141 2026-02-19T12:57:26.015 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 292057776141' 2026-02-19T12:57:26.015 INFO:tasks.workunit.client.0.trial144.stderr:220577: waiting osd.3 seq 292057776141 2026-02-19T12:57:26.015 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:57:26.015 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776142 -lt 292057776141 2026-02-19T12:57:26.015 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:26.015 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182422 2026-02-19T12:57:26.015 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:26.015 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:57:26.016 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-107374182422 2026-02-19T12:57:26.016 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:26.016 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182422 2026-02-19T12:57:26.016 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 107374182422' 2026-02-19T12:57:26.016 INFO:tasks.workunit.client.0.trial144.stderr:220577: waiting osd.4 seq 107374182422 2026-02-19T12:57:26.016 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:57:26.016 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182424 -lt 107374182422 2026-02-19T12:57:26.016 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:26.016 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-386547056642 2026-02-19T12:57:26.016 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:26.016 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:57:26.016 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-386547056642 2026-02-19T12:57:26.016 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:26.017 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=386547056642 2026-02-19T12:57:26.017 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 386547056642' 2026-02-19T12:57:26.017 INFO:tasks.workunit.client.0.trial144.stderr:220577: waiting osd.5 seq 386547056642 2026-02-19T12:57:26.017 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:57:26.017 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 386547056643 -lt 386547056642 2026-02-19T12:57:26.017 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:26.017 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-146028888084 2026-02-19T12:57:26.017 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:26.017 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:57:26.501 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/untu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:57:26.501 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-146028888085 2026-02-19T12:57:26.501 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:26.501 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888085 2026-02-19T12:57:26.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 146028888085' 2026-02-19T12:57:26.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: waiting osd.6 seq 146028888085 2026-02-19T12:57:26.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:57:26.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888085 -lt 146028888085 2026-02-19T12:57:26.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:26.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724564 2026-02-19T12:57:26.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:26.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:57:26.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-167503724564 2026-02-19T12:57:26.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:26.502 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724564 2026-02-19T12:57:26.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 167503724564' 2026-02-19T12:57:26.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: waiting osd.7 seq 167503724564 2026-02-19T12:57:26.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:57:26.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724565 -lt 167503724564 2026-02-19T12:57:26.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:26.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593748 2026-02-19T12:57:26.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:26.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:57:26.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-184683593748 2026-02-19T12:57:26.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:26.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593748 2026-02-19T12:57:26.503 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 184683593748' 2026-02-19T12:57:26.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: waiting osd.8 seq 184683593748 2026-02-19T12:57:26.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:57:26.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593748 -lt 184683593748 2026-02-19T12:57:26.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:26.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-386547056643 2026-02-19T12:57:26.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:26.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:57:26.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-386547056643 2026-02-19T12:57:26.504 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:26.518 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_ststandalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-146028888084 2026-02-19T12:57:26.518 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:26.518 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888084 2026-02-19T12:57:26.518 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 146028888084' 2026-02-19T12:57:26.518 INFO:tasks.workunit.client.0.trial144.stderr:220577: waiting osd.6 seq 146028888084 2026-02-19T12:57:26.518 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:57:26.518 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888085 -lt 146028888084 2026-02-19T12:57:26.519 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:26.519 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724563 2026-02-19T12:57:26.519 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:26.519 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:57:26.519 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-167503724563 2026-02-19T12:57:26.519 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:26.519 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724563 2026-02-19T12:57:26.519 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 167503724563' 2026-02-19T12:57:26.519 INFO:tasks.workunit.client.0.trial144.stderr:220577: waiting osd.7 seq 167503724563 2026-02-19T12:57:26.519 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:57:26.519 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724565 -lt 167503724563 2026-02-19T12:57:26.519 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:26.520 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593747 2026-02-19T12:57:26.520 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:26.520 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:57:26.520 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-184683593747 2026-02-19T12:57:26.520 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:26.520 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593747 2026-02-19T12:57:26.520 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 184683593747' 2026-02-19T12:57:26.520 INFO:tasks.workunit.client.0.trial144.stderr:220577: waiting osd.8 seq 184683593747 2026-02-19T12:57:26.520 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:57:26.520 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593748 -lt 184683593747 2026-02-19T12:57:26.520 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:26.520 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-386547056642 2026-02-19T12:57:26.520 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:26.521 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:57:26.521 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-386547056642 2026-02-19T12:57:26.521 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:26.521 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=386547056642 2026-02-19T12:57:27.199 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osats: seq=386547056643 2026-02-19T12:57:27.199 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 386547056643' 2026-02-19T12:57:27.199 INFO:tasks.workunit.client.0.trial144.stderr:220579: waiting osd.9 seq 386547056643 2026-02-19T12:57:27.199 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:57:27.199 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 386547056643 -lt 386547056643 2026-02-19T12:57:27.200 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:57:27.200 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:27.200 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:27.200 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:57:27.200 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:27.200 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:27.200 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:27.200 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:27.200 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:27.200 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:27.201 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:27.201 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:57:27.201 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:27.201 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:27.201 INFO:tasks.workunit.client.0.trial144.stderr:220579: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:27.201 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:57:27.201 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:57:27.201 INFO:tasks.workunit.client.0.trial144.stderr:220579: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:57:27.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:57:27.236 INFO:tasks.workunit.client.0.trial144.stderr:d.9 seq 386547056642' 2026-02-19T12:57:27.236 INFO:tasks.workunit.client.0.trial144.stderr:220577: waiting osd.9 seq 386547056642 2026-02-19T12:57:27.237 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:57:27.237 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 386547056643 -lt 386547056642 2026-02-19T12:57:27.237 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:57:27.237 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:27.237 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:27.237 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:57:27.237 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:27.237 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:27.237 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:27.237 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:27.238 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:27.238 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:27.238 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:27.238 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:57:27.238 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:27.238 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:27.238 INFO:tasks.workunit.client.0.trial144.stderr:220577: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:27.238 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:57:27.238 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:57:27.238 INFO:tasks.workunit.client.0.trial144.stderr:220577: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:57:27.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:57:27.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:57:27.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 220576 2026-02-19T12:57:27.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:57:27.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:57:27.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:57:27.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:257: corrupt_and_repair_two: return_code=0 2026-02-19T12:57:27.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:258: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:57:27.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-19T12:57:27.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:57:27.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:57:27.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:57:27.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:57:27.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: local pg=2.0 2026-02-19T12:57:27.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:264: corrupt_and_repair_two: repair 2.0 2026-02-19T12:57:27.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:57:27.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:57:27.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:27.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:27.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:27.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:27.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:57:11.513763+0000 2026-02-19T12:57:27.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:57:27.706 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:57:27.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:57:11.513763+0000 2026-02-19T12:57:27.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:57:27.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:57:11.513763+0000 2026-02-19T12:57:27.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:57:27.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:57:27.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:27.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:27.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:27.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:27.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:27.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:27.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:57:11.513763+0000 '>' 2026-02-19T12:57:11.513763+0000 2026-02-19T12:57:27.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:57:28.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:57:28.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:28.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:28.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:28.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:28.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:28.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:29.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:57:11.513763+0000 '>' 2026-02-19T12:57:11.513763+0000 2026-02-19T12:57:29.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:57:30.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:57:30.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:30.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:30.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:30.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:30.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:30.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:30.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:57:11.513763+0000 '>' 2026-02-19T12:57:11.513763+0000 2026-02-19T12:57:30.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:57:31.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:57:31.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:31.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:31.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:31.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:31.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:31.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:31.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:57:11.513763+0000 '>' 2026-02-19T12:57:11.513763+0000 2026-02-19T12:57:31.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:57:32.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:57:32.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:32.357 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:32.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:32.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:32.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:32.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:32.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:57:11.513763+0000 '>' 2026-02-19T12:57:11.513763+0000 2026-02-19T12:57:32.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:57:33.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:57:33.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:33.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:33.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:33.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:33.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:33.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:33.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:57:11.513763+0000 '>' 2026-02-19T12:57:11.513763+0000 2026-02-19T12:57:33.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:57:34.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:57:34.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:34.673 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:34.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:34.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:34.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:34.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:34.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:57:28.446265+0000 '>' 2026-02-19T12:57:11.513763+0000 2026-02-19T12:57:34.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:57:34.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:268: corrupt_and_repair_two: pids= 2026-02-19T12:57:34.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:269: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:57:34.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 224036"' 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 224036' 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:270: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 224038"' 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 224038' 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:271: corrupt_and_repair_two: wait_background pids 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 224036 224038' 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 224036 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/224039: /' 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:57:34.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/224041: /' 2026-02-19T12:57:35.330 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:57:35.331 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:57:35.331 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-19T12:57:35.331 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:57:35.331 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:57:35.331 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:57:35.331 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:57:35.331 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-19T12:57:35.331 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:57:35.331 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-19T12:57:35.331 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:57:35.331 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:57:35.331 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:57:35.331 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING list-attrs 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: _ 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: hinfo_key 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: snapset 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:57:35.332 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-19T12:57:35.333 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:57:35.333 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:57:35.333 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:57:35.333 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:57:35.333 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:57:35.333 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:57:35.333 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-19T12:57:35.333 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:57:35.333 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:57:35.334 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:57:35.334 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:57:35.334 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-19T12:57:35.334 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:57:35.334 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-19T12:57:35.334 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:57:35.334 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:57:35.334 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:57:35.334 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:57:35.334 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:57:35.334 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:57:35.334 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-19T12:57:35.334 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:57:35.335 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:57:35.335 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-19T12:57:35.335 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:57:35.335 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:57:35.335 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-19T12:57:35.335 INFO:tasks.workunit.client.0.trial144.stderr:224041: _ 2026-02-19T12:57:35.335 INFO:tasks.workunit.client.0.trial144.stderr:224041: hinfo_key 2026-02-19T12:57:35.335 INFO:tasks.workunit.client.0.trial144.stderr:224041: snapset 2026-02-19T12:57:35.335 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-19T12:57:35.335 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:57:35.336 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:57:35.336 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-19T12:57:35.336 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:57:35.336 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-19T12:57:35.336 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:57:35.336 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:57:35.337 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:57:35.337 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:57:35.337 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-19T12:57:35.337 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-19T12:57:35.337 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:57:35.337 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:57:35.337 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:57:35.337 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: start osd.5 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recos.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:57:35.338 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-19T12:57:35.339 INFO:tasks.workunit.client.0.trial144.stderr:224041: start osd.9 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: 2026-02-19T12:57:35.348+0000 7fac30175a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: 2026-02-19T12:57:35.352+0000 7fac30175a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: 2026-02-19T12:57:35.352+0000 7fac30175a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: 0 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:57:36.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: 2026-02-19T12:57:35.780+0000 7fac30175a40 -1 Falling back to public interface 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: 2026-02-19T12:57:36.508+0000 7fac30175a40 -1 osd.5 92 log_to_monitors true 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: 1 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: osd.5 up in weight 1 up_from 96 up_thru 96 down_at 93 last_clean_interval [90,92) [v2:127.0.0.1:6842/1468639031,v1:127.0.0.1:6843/1468639031] [v2:127.0.0.1:6844/1468639031,v1:127.0.0.1:6845/1468639031] exists,up f2b41730-2834-41c5-a57f-2a8a981fe2c0 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:57:36.818 INFO:tasks.workunit.client.0.trial144.stderr:224039: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:57:36.823 INFO:tasks.workunit.client.0.trial144.stderr:224039: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.very_ops 2026-02-19T12:57:36.823 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-19T12:57:36.823 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-19T12:57:36.823 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:57:36.823 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:57:36.823 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:57:36.823 INFO:tasks.workunit.client.0.trial144.stderr:224041: 2026-02-19T12:57:35.348+0000 7fdeefbbca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:36.824 INFO:tasks.workunit.client.0.trial144.stderr:224041: 2026-02-19T12:57:35.352+0000 7fdeefbbca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:36.824 INFO:tasks.workunit.client.0.trial144.stderr:224041: 2026-02-19T12:57:35.352+0000 7fdeefbbca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:36.824 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-19T12:57:36.824 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:57:36.824 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-19T12:57:36.824 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:57:36.824 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:57:36.824 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:36.824 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:57:36.824 INFO:tasks.workunit.client.0.trial144.stderr:224041: 0 2026-02-19T12:57:36.824 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:36.824 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:57:36.824 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:57:36.825 INFO:tasks.workunit.client.0.trial144.stderr:224041: 2026-02-19T12:57:35.816+0000 7fdeefbbca40 -1 Falling back to public interface 2026-02-19T12:57:36.825 INFO:tasks.workunit.client.0.trial144.stderr:224041: 2026-02-19T12:57:36.544+0000 7fdeefbbca40 -1 osd.9 92 log_to_monitors true 2026-02-19T12:57:36.825 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:57:36.825 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:36.825 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:57:36.825 INFO:tasks.workunit.client.0.trial144.stderr:224041: 1 2026-02-19T12:57:36.825 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:36.825 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T12:57:36.825 INFO:tasks.workunit.client.0.trial144.stderr:224041: osd.9 up in weight 1 up_from 96 up_thru 0 down_at 93 last_clean_interval [90,92) [v2:127.0.0.1:6874/3800742223,v1:127.0.0.1:6875/3800742223] [v2:127.0.0.1:6876/3800742223,v1:127.0.0.1:6877/3800742223] exists,up 9f6223c2-19a3-4d20-afdc-51e479427486 2026-02-19T12:57:36.825 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:57:36.825 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:57:36.825 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:57:36.825 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:57:36.826 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:57:36.826 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:57:36.826 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:57:36.826 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:57:36.826 INFO:tasks.workunit.client.0.trial144.stderr:224041: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224041: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.ssh:1593: get_timeout_delays: echo true 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: 1 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: 2 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: 3 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: 4 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: 5 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: 6 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: 7 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: 8 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: 9' 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836509 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836509 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509' 2026-02-19T12:57:37.370 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.371 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:57:37.371 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672988 2026-02-19T12:57:37.371 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672988 2026-02-19T12:57:37.371 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-42949672988' 2026-02-19T12:57:37.371 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.371 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:57:37.371 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509468 2026-02-19T12:57:37.371 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509468 2026-02-19T12:57:37.371 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509468' 2026-02-19T12:57:37.371 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.371 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:57:37.371 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776145 2026-02-19T12:57:37.371 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776145 2026-02-19T12:57:37.371 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509468 3-292057776145' 2026-02-19T12:57:37.371 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helperh:1593: get_timeout_delays: echo true 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: 1 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: 2 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: 3 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: 4 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: 5 2026-02-19T12:57:37.378 INFO:tasks.workunit.client.0.trial144.stderr:224041: 6 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: 7 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: 8 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: 9' 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836510 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836510 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510' 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672989 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672989 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-42949672989' 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509469 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509469 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-42949672989 2-64424509469' 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776146 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776146 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-42949672989 2-64424509469 3-292057776146' 2026-02-19T12:57:37.379 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.814 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:57:37.814 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182428 2026-02-19T12:57:37.814 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182428 2026-02-19T12:57:37.814 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-42949672989 2-64424509469 3-292057776146 4-107374182428' 2026-02-19T12:57:37.814 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.814 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:57:37.814 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=412316860419 2026-02-19T12:57:37.814 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 412316860419 2026-02-19T12:57:37.814 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-42949672989 2-64424509469 3-292057776146 4-107374182428 5-412316860419' 2026-02-19T12:57:37.814 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.814 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:57:37.814 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888089 2026-02-19T12:57:37.814 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888089 2026-02-19T12:57:37.814 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-42949672989 2-64424509469 3-292057776146 4-107374182428 5-412316860419 6-146028888089' 2026-02-19T12:57:37.814 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.815 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:57:37.815 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724568 2026-02-19T12:57:37.815 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724568 2026-02-19T12:57:37.815 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-42949672989 2-64424509469 3-292057776146 4-107374182428 5-412316860419 6-146028888089 7-167503724568' 2026-02-19T12:57:37.815 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.815 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:57:37.815 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593751 2026-02-19T12:57:37.815 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593751 2026-02-19T12:57:37.815 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-42949672989 2-64424509469 3-292057776146 4-107374182428 5-412316860419 6-146028888089 7-167503724568 8-184683593751' 2026-02-19T12:57:37.815 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.815 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:57:37.815 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=412316860418 2026-02-19T12:57:37.815 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 412316860418 2026-02-19T12:57:37.815 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836510 1-42949672989 2-64424509469 3-292057776146 4-107374182428 5-412316860419 6-146028888089 7-167503724568 8-184683593751 9-412316860418' 2026-02-19T12:57:37.815 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224041: s.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182427 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182427 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509468 3-292057776145 4-107374182427' 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=412316860418 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 412316860418 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509468 3-292057776145 4-107374182427 5-412316860418' 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888090 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888090 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509468 3-292057776145 4-107374182427 5-412316860418 6-146028888090' 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724569 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724569 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509468 3-292057776145 4-107374182427 5-412316860418 6-146028888090 7-167503724569' 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593752 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593752 2026-02-19T12:57:37.816 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509468 3-292057776145 4-107374182427 5-412316860418 6-146028888090 7-167503724569 8-184683593752' 2026-02-19T12:57:37.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:37.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:57:37.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=412316860419 2026-02-19T12:57:37.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 412316860419 2026-02-19T12:57:37.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509468 3-292057776145 4-107374182427 5-412316860418 6-146028888090 7-167503724569 8-184683593752 9-412316860419' 2026-02-19T12:57:37.817 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:39.463 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836509 2026-02-19T12:57:39.463 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:39.463 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:57:39.463 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836509 2026-02-19T12:57:39.463 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:39.463 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836509 2026-02-19T12:57:39.463 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836509' 2026-02-19T12:57:39.463 INFO:tasks.workunit.client.0.trial144.stderr:224039: waiting osd.0 seq 21474836509 2026-02-19T12:57:39.464 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:39.464 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836508 -lt 21474836509 2026-02-19T12:57:39.464 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:57:39.464 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:57:39.464 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:39.464 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836510 -lt 21474836509 2026-02-19T12:57:39.464 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:39.464 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672988 2026-02-19T12:57:39.464 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:39.464 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:57:39.464 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672988 2026-02-19T12:57:39.464 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:39.464 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672988 2026-02-19T12:57:39.465 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672988' 2026-02-19T12:57:39.465 INFO:tasks.workunit.client.0.trial144.stderr:224039: waiting osd.1 seq 42949672988 2026-02-19T12:57:39.465 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:57:39.465 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672990 -lt 42949672988 2026-02-19T12:57:39.465 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:39.465 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509468 2026-02-19T12:57:39.465 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:39.465 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:57:39.465 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509468 2026-02-19T12:57:39.465 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:39.465 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509468 2026-02-19T12:57:39.465 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509468' 2026-02-19T12:57:39.466 INFO:tasks.workunit.client.0.trial144.stderr:224039: waiting osd.2 seq 64424509468 2026-02-19T12:57:39.466 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:57:39.466 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509469 -lt 64424509468 2026-02-19T12:57:39.472 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2//home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836510 2026-02-19T12:57:39.472 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:39.472 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:57:39.472 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836510 2026-02-19T12:57:39.472 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:39.472 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836510 2026-02-19T12:57:39.472 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836510' 2026-02-19T12:57:39.472 INFO:tasks.workunit.client.0.trial144.stderr:224041: waiting osd.0 seq 21474836510 2026-02-19T12:57:39.473 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:39.473 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836508 -lt 21474836510 2026-02-19T12:57:39.473 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:57:39.473 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:57:39.473 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:39.473 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836510 -lt 21474836510 2026-02-19T12:57:39.473 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:39.473 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672989 2026-02-19T12:57:39.473 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:39.473 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:57:39.473 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672989 2026-02-19T12:57:39.473 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:39.474 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672989 2026-02-19T12:57:39.474 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672989' 2026-02-19T12:57:39.474 INFO:tasks.workunit.client.0.trial144.stderr:224041: waiting osd.1 seq 42949672989 2026-02-19T12:57:39.474 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:57:39.474 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672990 -lt 42949672989 2026-02-19T12:57:39.474 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:39.474 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509469 2026-02-19T12:57:39.474 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:39.474 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:57:39.474 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509469 2026-02-19T12:57:39.474 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:39.475 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509469 2026-02-19T12:57:39.475 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509469' 2026-02-19T12:57:39.475 INFO:tasks.workunit.client.0.trial144.stderr:224041: waiting osd.2 seq 64424509469 2026-02-19T12:57:39.475 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:57:39.475 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509469 -lt 64424509469 2026-02-19T12:57:39.953 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:22266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:39.953 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-292057776145 2026-02-19T12:57:39.953 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:39.953 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:57:39.953 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-292057776145 2026-02-19T12:57:39.954 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:39.954 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776145 2026-02-19T12:57:39.954 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 292057776145' 2026-02-19T12:57:39.954 INFO:tasks.workunit.client.0.trial144.stderr:224039: waiting osd.3 seq 292057776145 2026-02-19T12:57:39.954 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:57:39.954 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776147 -lt 292057776145 2026-02-19T12:57:39.954 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:39.954 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182427 2026-02-19T12:57:39.954 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:39.955 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:57:39.955 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-107374182427 2026-02-19T12:57:39.955 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:39.955 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182427 2026-02-19T12:57:39.955 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 107374182427' 2026-02-19T12:57:39.955 INFO:tasks.workunit.client.0.trial144.stderr:224039: waiting osd.4 seq 107374182427 2026-02-19T12:57:39.955 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:57:39.955 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182428 -lt 107374182427 2026-02-19T12:57:39.955 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:39.955 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-412316860418 2026-02-19T12:57:39.955 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:39.955 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:57:39.956 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-412316860418 2026-02-19T12:57:39.956 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:39.956 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=412316860418 2026-02-19T12:57:39.956 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 412316860418' 2026-02-19T12:57:39.956 INFO:tasks.workunit.client.0.trial144.stderr:224039: waiting osd.5 seq 412316860418 2026-02-19T12:57:39.956 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:57:39.956 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 412316860419 -lt 412316860418 2026-02-19T12:57:39.956 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:39.956 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-146028888090 2026-02-19T12:57:39.956 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:39.956 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:57:39.974 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.cl66: flush_pg_stats: for s in $seqs 2026-02-19T12:57:39.975 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-292057776146 2026-02-19T12:57:39.975 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:39.975 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:57:39.975 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-292057776146 2026-02-19T12:57:39.975 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:39.975 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776146 2026-02-19T12:57:39.975 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 292057776146' 2026-02-19T12:57:39.975 INFO:tasks.workunit.client.0.trial144.stderr:224041: waiting osd.3 seq 292057776146 2026-02-19T12:57:39.975 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:57:39.976 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776147 -lt 292057776146 2026-02-19T12:57:39.976 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:39.976 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182428 2026-02-19T12:57:39.976 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:39.976 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:57:39.976 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-107374182428 2026-02-19T12:57:39.976 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:39.976 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182428 2026-02-19T12:57:39.976 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 107374182428' 2026-02-19T12:57:39.976 INFO:tasks.workunit.client.0.trial144.stderr:224041: waiting osd.4 seq 107374182428 2026-02-19T12:57:39.976 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:57:39.976 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182428 -lt 107374182428 2026-02-19T12:57:39.977 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:39.977 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-412316860419 2026-02-19T12:57:39.977 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:39.977 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:57:39.977 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-412316860419 2026-02-19T12:57:39.977 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:39.977 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=412316860419 2026-02-19T12:57:39.977 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 412316860419' 2026-02-19T12:57:39.977 INFO:tasks.workunit.client.0.trial144.stderr:224041: waiting osd.5 seq 412316860419 2026-02-19T12:57:39.977 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:57:39.977 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 412316860419 -lt 412316860419 2026-02-19T12:57:39.977 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:39.978 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-146028888089 2026-02-19T12:57:39.978 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:39.978 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:57:40.440 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.cliient.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-146028888090 2026-02-19T12:57:40.440 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:40.440 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888090 2026-02-19T12:57:40.440 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 146028888090' 2026-02-19T12:57:40.440 INFO:tasks.workunit.client.0.trial144.stderr:224039: waiting osd.6 seq 146028888090 2026-02-19T12:57:40.440 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:57:40.441 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888090 -lt 146028888090 2026-02-19T12:57:40.441 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:40.441 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724569 2026-02-19T12:57:40.441 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:40.441 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:57:40.441 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-167503724569 2026-02-19T12:57:40.441 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:40.441 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724569 2026-02-19T12:57:40.441 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 167503724569' 2026-02-19T12:57:40.441 INFO:tasks.workunit.client.0.trial144.stderr:224039: waiting osd.7 seq 167503724569 2026-02-19T12:57:40.441 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:57:40.442 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724570 -lt 167503724569 2026-02-19T12:57:40.442 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:40.442 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593752 2026-02-19T12:57:40.442 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:40.442 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:57:40.442 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-184683593752 2026-02-19T12:57:40.442 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:40.442 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593752 2026-02-19T12:57:40.442 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 184683593752' 2026-02-19T12:57:40.442 INFO:tasks.workunit.client.0.trial144.stderr:224039: waiting osd.8 seq 184683593752 2026-02-19T12:57:40.442 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:57:40.442 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593753 -lt 184683593752 2026-02-19T12:57:40.443 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:40.443 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-412316860419 2026-02-19T12:57:40.443 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:40.443 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:57:40.443 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-412316860419 2026-02-19T12:57:40.443 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:40.443 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=412316860419 2026-02-19T12:57:40.489 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'ent.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-146028888089 2026-02-19T12:57:40.489 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:40.489 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888089 2026-02-19T12:57:40.490 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 146028888089' 2026-02-19T12:57:40.490 INFO:tasks.workunit.client.0.trial144.stderr:224041: waiting osd.6 seq 146028888089 2026-02-19T12:57:40.490 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:57:40.490 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888090 -lt 146028888089 2026-02-19T12:57:40.490 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:40.490 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724568 2026-02-19T12:57:40.490 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:40.490 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:57:40.490 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-167503724568 2026-02-19T12:57:40.490 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:40.490 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724568 2026-02-19T12:57:40.491 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 167503724568' 2026-02-19T12:57:40.491 INFO:tasks.workunit.client.0.trial144.stderr:224041: waiting osd.7 seq 167503724568 2026-02-19T12:57:40.491 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:57:40.491 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724570 -lt 167503724568 2026-02-19T12:57:40.491 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:40.491 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593751 2026-02-19T12:57:40.491 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:40.491 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:57:40.491 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-184683593751 2026-02-19T12:57:40.491 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:40.491 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593751 2026-02-19T12:57:40.491 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 184683593751' 2026-02-19T12:57:40.491 INFO:tasks.workunit.client.0.trial144.stderr:224041: waiting osd.8 seq 184683593751 2026-02-19T12:57:40.492 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:57:40.492 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593753 -lt 184683593751 2026-02-19T12:57:40.492 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:40.492 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-412316860418 2026-02-19T12:57:40.492 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:40.492 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:57:40.492 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-412316860418 2026-02-19T12:57:40.492 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:40.492 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=412316860418 2026-02-19T12:57:41.421 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'wwaiting osd.9 seq 412316860419' 2026-02-19T12:57:41.421 INFO:tasks.workunit.client.0.trial144.stderr:224039: waiting osd.9 seq 412316860419 2026-02-19T12:57:41.422 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:57:41.422 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 412316860419 -lt 412316860419 2026-02-19T12:57:41.422 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:57:41.422 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:41.422 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:41.422 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:57:41.423 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:41.423 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:41.423 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:41.423 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:41.423 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:41.423 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:41.423 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:41.424 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:57:41.424 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:41.424 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:41.424 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:41.424 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:57:41.424 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-19T12:57:41.424 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T12:57:41.425 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-19T12:57:41.425 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:57:41.425 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T12:57:41.425 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:57:41.425 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:41.425 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:41.425 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:41.426 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:41.426 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:41.426 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:41.426 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:41.475 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubunaiting osd.9 seq 412316860418' 2026-02-19T12:57:41.476 INFO:tasks.workunit.client.0.trial144.stderr:224041: waiting osd.9 seq 412316860418 2026-02-19T12:57:41.476 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:57:41.476 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 412316860419 -lt 412316860418 2026-02-19T12:57:41.476 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:57:41.476 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:41.476 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:41.477 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:57:41.477 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:41.477 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:41.477 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:41.477 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:41.477 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:41.477 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:41.478 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:41.478 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:57:41.478 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:41.478 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:41.478 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:41.478 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:57:41.479 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-19T12:57:41.479 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T12:57:41.479 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-19T12:57:41.479 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:57:41.479 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T12:57:41.479 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:57:41.479 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:41.479 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:41.480 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:41.480 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:41.480 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:41.480 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:41.480 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:42.382 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubunttu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:57:42.382 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:42.382 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:42.382 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:42.382 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:57:42.382 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T12:57:42.382 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:57:42.382 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:57:42.382 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:57:42.382 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:57:42.382 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:57:42.382 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:42.383 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:57:42.401 INFO:tasks.workunit.client.0.trial144.stderr:224u/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:57:42.401 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:42.401 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:42.401 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:42.401 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:57:42.401 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:42.402 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:42.403 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:42.403 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:42.403 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:42.403 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:42.403 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:57:42.403 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:42.403 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:42.403 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:42.403 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:2240039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:43.340 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:57:43.341 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T12:57:43.341 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:57:43.341 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:57:43.341 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:57:43.342 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ub41: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T12:57:43.342 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:57:43.342 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:57:43.342 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:57:43.342 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:57:43.342 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:57:43.342 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:57:43.342 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:57:43.342 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:43.343 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:43.344 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:57:43.344 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T12:57:43.344 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:57:43.344 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:57:43.344 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:57:44.698 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:57:44.698 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:57:44.698 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:57:44.698 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:57:44.698 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:57:44.698 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:57:44.698 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 13 )) 2026-02-19T12:57:44.699 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:57:44.699 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-19T12:57:44.699 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:57:44.699 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:44.699 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:44.699 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:44.699 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:44.699 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:44.699 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:44.699 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:44.699 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:57:44.700 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:44.700 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:44.700 INFO:tasks.workunit.client.0.trial144.stderr:224039: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:44.700 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:57:44.700 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:57:44.700 INFO:tasks.workunit.client.0.trial144.stderr:224039: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:57:44.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:57:44.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:57:44.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 224038 2026-02-19T12:57:44.710 INFO:tasks.workunit.client.0.trial144.stderr:ntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:57:44.710 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:57:44.710 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:57:44.711 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:57:44.711 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:57:44.711 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:57:44.711 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 13 )) 2026-02-19T12:57:44.711 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:57:44.711 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-19T12:57:44.711 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:57:44.711 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:44.711 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:44.711 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:44.711 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:44.712 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:44.712 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:44.712 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:44.712 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:57:44.712 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:44.712 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:44.712 INFO:tasks.workunit.client.0.trial144.stderr:224041: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:44.712 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:57:44.712 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:57:44.712 INFO:tasks.workunit.client.0.trial144.stderr:224041: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:57:44.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:57:44.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:57:44.714 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:57:44.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:57:44.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:272: corrupt_and_repair_two: return_code=0 2026-02-19T12:57:44.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:273: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:57:44.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:275: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:57:44.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:276: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:57:44.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:323: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 3 5 2026-02-19T12:57:44.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:245: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-19T12:57:44.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:246: corrupt_and_repair_two: local poolname=ecpool 2026-02-19T12:57:44.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:247: corrupt_and_repair_two: local first=3 2026-02-19T12:57:44.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_two: local second=5 2026-02-19T12:57:44.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_two: pids= 2026-02-19T12:57:44.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:57:44.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:57:44.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:57:44.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 227445"' 2026-02-19T12:57:44.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 227445' 2026-02-19T12:57:44.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:57:44.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:57:44.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:57:44.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:57:44.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 227447"' 2026-02-19T12:57:44.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 227447' 2026-02-19T12:57:44.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_two: wait_background pids 2026-02-19T12:57:44.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 227445 227447' 2026-02-19T12:57:44.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:57:44.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:57:44.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 227445 2026-02-19T12:57:44.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/227448: /' 2026-02-19T12:57:44.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:57:44.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/227451: /' 2026-02-19T12:57:45.355 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:57:45.355 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:57:45.355 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:57:45.355 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:57:45.355 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:57:45.355 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:57:45.356 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:57:45.356 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:57:45.356 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:57:45.356 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:57:45.356 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:57:45.356 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:57:45.356 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:57:45.356 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:57:45.356 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:57:45.356 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:57:45.356 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T12:57:45.356 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:57:45.357 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:57:45.357 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:57:45.357 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:57:45.357 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:57:45.357 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-19T12:57:45.357 INFO:tasks.workunit.client.0.trial144.stderr:227448: remove 0#2:eb822e21:::SOMETHING:head# 2026-02-19T12:57:45.357 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:57:45.357 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:57:45.357 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:57:45.357 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:57:45.357 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:57:45.357 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:57:45.358 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:57:45.358 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:57:45.358 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:57:45.358 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:57:45.359 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:57:45.359 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:57:45.359 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:57:45.359 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:57:45.359 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:57:45.359 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:57:45.359 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:57:45.359 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:57:45.359 INFO:tasks.workunit.client.0.trial144.stderr:227448: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:57:45.359 INFO:tasks.workunit.client.0.trial144.stderr:227448: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:57:45.359 INFO:tasks.workunit.client.0.trial144.stderr:227448: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:57:45.359 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:45.360 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:45.360 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:57:45.360 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:57:45.360 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:57:45.360 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:57:45.360 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:57:45.360 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:57:45.360 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:57:45.360 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:57:45.360 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:57:45.360 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:57:45.360 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:57:45.361 INFO:tasks.workunit.client.0.trial144.stderr:227448: start osd.3 2026-02-19T12:57:45.361 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:57:45.383 INFO:tasks.workunit.client.0.trial144.stderr:2227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:57:45.383 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:57:45.383 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-19T12:57:45.384 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:57:45.384 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:57:45.384 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:57:45.384 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:57:45.384 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-19T12:57:45.384 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:57:45.384 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-19T12:57:45.384 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:57:45.384 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:57:45.384 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:57:45.384 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:57:45.384 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:57:45.385 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:57:45.385 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-19T12:57:45.385 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:57:45.385 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:57:45.385 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-19T12:57:45.385 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:57:45.385 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:57:45.385 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-19T12:57:45.385 INFO:tasks.workunit.client.0.trial144.stderr:227451: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-19T12:57:45.385 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-19T12:57:45.385 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:57:45.385 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:57:45.385 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-19T12:57:45.386 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:57:45.386 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:57:45.386 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:57:45.386 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:57:45.387 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:57:45.387 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:57:45.387 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-19T12:57:45.387 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-19T12:57:45.387 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:57:45.387 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:57:45.388 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:57:45.388 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:57:45.388 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:57:45.388 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:57:45.388 INFO:tasks.workunit.client.0.trial144.stderr:227451: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:57:45.388 INFO:tasks.workunit.client.0.trial144.stderr:227451: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:57:45.388 INFO:tasks.workunit.client.0.trial144.stderr:227451: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:57:45.388 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:45.388 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:45.388 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:57:45.388 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:57:45.388 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:57:45.389 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:57:45.389 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:57:45.389 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:57:45.389 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:57:45.389 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:57:45.389 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:57:45.389 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-19T12:57:45.389 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-19T12:57:45.389 INFO:tasks.workunit.client.0.trial144.stderr:227451: start osd.5 2026-02-19T12:57:45.389 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:57:46.852 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:57:46.853 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:57:46.853 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:57:46.853 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:57:46.853 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:57:46.853 INFO:tasks.workunit.client.0.trial144.stderr:227448: 2026-02-19T12:57:45.384+0000 7f552be66a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:46.853 INFO:tasks.workunit.client.0.trial144.stderr:227448: 2026-02-19T12:57:45.384+0000 7f552be66a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:46.853 INFO:tasks.workunit.client.0.trial144.stderr:227448: 2026-02-19T12:57:45.384+0000 7f552be66a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:46.853 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:57:46.853 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:57:46.853 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:57:46.854 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:57:46.854 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:57:46.854 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:46.854 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:57:46.854 INFO:tasks.workunit.client.0.trial144.stderr:227448: 0 2026-02-19T12:57:46.854 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:46.854 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:57:46.854 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:57:46.854 INFO:tasks.workunit.client.0.trial144.stderr:227448: 2026-02-19T12:57:45.820+0000 7f552be66a40 -1 Falling back to public interface 2026-02-19T12:57:46.854 INFO:tasks.workunit.client.0.trial144.stderr:227448: 2026-02-19T12:57:46.516+0000 7f552be66a40 -1 osd.3 98 log_to_monitors true 2026-02-19T12:57:46.854 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:57:46.854 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:46.855 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:57:46.855 INFO:tasks.workunit.client.0.trial144.stderr:227448: 1 2026-02-19T12:57:46.855 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:46.855 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:57:46.855 INFO:tasks.workunit.client.0.trial144.stderr:227448: osd.3 up in weight 1 up_from 101 up_thru 101 down_at 98 last_clean_interval [68,98) [v2:127.0.0.1:6826/1542882463,v1:127.0.0.1:6827/1542882463] [v2:127.0.0.1:6828/1542882463,v1:127.0.0.1:6829/1542882463] exists,up ef3f1d22-1abd-4877-a9fc-b7ff790fe73e 2026-02-19T12:57:46.855 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:57:46.855 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:57:46.855 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:57:46.855 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:57:46.855 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:57:46.855 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:57:46.855 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:57:46.856 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:57:46.856 INFO:tasks.workunit.client.0.trial144.stderr:227448: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:57:46.880 INFO:tasks.workunit.client.0.trial144.stderr:227448: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593:27451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-19T12:57:46.880 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-19T12:57:46.880 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:57:46.880 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:57:46.880 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:57:46.880 INFO:tasks.workunit.client.0.trial144.stderr:227451: 2026-02-19T12:57:45.400+0000 7fdfcf5dca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:46.880 INFO:tasks.workunit.client.0.trial144.stderr:227451: 2026-02-19T12:57:45.404+0000 7fdfcf5dca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:46.880 INFO:tasks.workunit.client.0.trial144.stderr:227451: 2026-02-19T12:57:45.404+0000 7fdfcf5dca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:57:46.880 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-19T12:57:46.881 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:57:46.881 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T12:57:46.881 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:57:46.881 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:57:46.881 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:46.881 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:57:46.881 INFO:tasks.workunit.client.0.trial144.stderr:227451: 0 2026-02-19T12:57:46.881 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:46.881 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:57:46.881 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:57:46.881 INFO:tasks.workunit.client.0.trial144.stderr:227451: 2026-02-19T12:57:45.836+0000 7fdfcf5dca40 -1 Falling back to public interface 2026-02-19T12:57:46.881 INFO:tasks.workunit.client.0.trial144.stderr:227451: 2026-02-19T12:57:46.540+0000 7fdfcf5dca40 -1 osd.5 97 log_to_monitors true 2026-02-19T12:57:46.882 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:57:46.882 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:57:46.882 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:57:46.882 INFO:tasks.workunit.client.0.trial144.stderr:227451: 1 2026-02-19T12:57:46.882 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:57:46.882 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:57:46.882 INFO:tasks.workunit.client.0.trial144.stderr:227451: osd.5 up in weight 1 up_from 101 up_thru 101 down_at 98 last_clean_interval [96,97) [v2:127.0.0.1:6842/3158370433,v1:127.0.0.1:6843/3158370433] [v2:127.0.0.1:6844/3158370433,v1:127.0.0.1:6845/3158370433] exists,up f2b41730-2834-41c5-a57f-2a8a981fe2c0 2026-02-19T12:57:46.882 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:57:46.882 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:57:46.882 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:57:46.882 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:57:46.882 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:57:46.882 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:57:46.883 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:57:46.883 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:57:46.883 INFO:tasks.workunit.client.0.trial144.stderr:227451: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227451: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: 1 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: 2 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: 3 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: 4 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: 5 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: 6 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: 7 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: 8 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: 9' 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836513 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836513 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513' 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672992 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672992 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-42949672992' 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509472 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509472 2026-02-19T12:57:47.415 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-42949672992 2-64424509472' 2026-02-19T12:57:47.416 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.416 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:57:47.416 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=433791696898 2026-02-19T12:57:47.416 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 433791696898 2026-02-19T12:57:47.416 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-42949672992 2-64424509472 3-433791696898' 2026-02-19T12:57:47.416 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:225 get_timeout_delays: echo true 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: 1 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: 2 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: 3 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: 4 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: 5 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: 6 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: 7 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: 8 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: 9' 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836514 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836514 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514' 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.428 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:57:47.429 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672993 2026-02-19T12:57:47.429 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672993 2026-02-19T12:57:47.429 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672993' 2026-02-19T12:57:47.429 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.429 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:57:47.429 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509473 2026-02-19T12:57:47.429 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509473 2026-02-19T12:57:47.429 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-64424509473' 2026-02-19T12:57:47.429 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.429 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:57:47.429 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=433791696899 2026-02-19T12:57:47.429 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 433791696899 2026-02-19T12:57:47.429 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-64424509473 3-433791696899' 2026-02-19T12:57:47.429 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.859 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:57:47.860 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182431 2026-02-19T12:57:47.860 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182431 2026-02-19T12:57:47.860 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-42949672992 2-64424509472 3-433791696898 4-107374182431' 2026-02-19T12:57:47.860 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.860 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:57:47.861 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=433791696898 2026-02-19T12:57:47.861 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 433791696898 2026-02-19T12:57:47.861 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-42949672992 2-64424509472 3-433791696898 4-107374182431 5-433791696898' 2026-02-19T12:57:47.861 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.861 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:57:47.861 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888093 2026-02-19T12:57:47.861 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888093 2026-02-19T12:57:47.862 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-42949672992 2-64424509472 3-433791696898 4-107374182431 5-433791696898 6-146028888093' 2026-02-19T12:57:47.862 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.862 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:57:47.862 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724572 2026-02-19T12:57:47.862 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724572 2026-02-19T12:57:47.862 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-42949672992 2-64424509472 3-433791696898 4-107374182431 5-433791696898 6-146028888093 7-167503724572' 2026-02-19T12:57:47.862 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.863 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:57:47.863 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593755 2026-02-19T12:57:47.863 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593755 2026-02-19T12:57:47.863 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-42949672992 2-64424509472 3-433791696898 4-107374182431 5-433791696898 6-146028888093 7-167503724572 8-184683593755' 2026-02-19T12:57:47.863 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.863 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:57:47.863 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=412316860423 2026-02-19T12:57:47.863 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 412316860423 2026-02-19T12:57:47.864 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836513 1-42949672992 2-64424509472 3-433791696898 4-107374182431 5-433791696898 6-146028888093 7-167503724572 8-184683593755 9-412316860423' 2026-02-19T12:57:47.864 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:47.864 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/8: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:57:47.865 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182432 2026-02-19T12:57:47.865 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182432 2026-02-19T12:57:47.865 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-64424509473 3-433791696899 4-107374182432' 2026-02-19T12:57:47.865 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.865 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:57:47.865 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=433791696899 2026-02-19T12:57:47.865 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 433791696899 2026-02-19T12:57:47.865 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-64424509473 3-433791696899 4-107374182432 5-433791696899' 2026-02-19T12:57:47.865 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.866 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:57:47.866 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888094 2026-02-19T12:57:47.866 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888094 2026-02-19T12:57:47.866 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-64424509473 3-433791696899 4-107374182432 5-433791696899 6-146028888094' 2026-02-19T12:57:47.866 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.866 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:57:47.866 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724573 2026-02-19T12:57:47.866 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724573 2026-02-19T12:57:47.866 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-64424509473 3-433791696899 4-107374182432 5-433791696899 6-146028888094 7-167503724573' 2026-02-19T12:57:47.866 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.866 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:57:47.867 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593756 2026-02-19T12:57:47.867 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593756 2026-02-19T12:57:47.867 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-64424509473 3-433791696899 4-107374182432 5-433791696899 6-146028888094 7-167503724573 8-184683593756' 2026-02-19T12:57:47.867 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:57:47.867 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:57:47.867 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=412316860422 2026-02-19T12:57:47.867 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 412316860422 2026-02-19T12:57:47.867 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-64424509473 3-433791696899 4-107374182432 5-433791696899 6-146028888094 7-167503724573 8-184683593756 9-412316860422' 2026-02-19T12:57:47.867 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:49.521 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836514 2026-02-19T12:57:49.521 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:49.521 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:57:49.521 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836514 2026-02-19T12:57:49.521 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:49.521 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836514 2026-02-19T12:57:49.521 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836514' 2026-02-19T12:57:49.521 INFO:tasks.workunit.client.0.trial144.stderr:227451: waiting osd.0 seq 21474836514 2026-02-19T12:57:49.522 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:49.522 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836512 -lt 21474836514 2026-02-19T12:57:49.522 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:57:49.522 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:57:49.522 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:49.522 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836514 -lt 21474836514 2026-02-19T12:57:49.522 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:49.522 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672993 2026-02-19T12:57:49.522 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:49.522 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:57:49.522 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672993 2026-02-19T12:57:49.522 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:49.522 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672993 2026-02-19T12:57:49.523 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672993' 2026-02-19T12:57:49.523 INFO:tasks.workunit.client.0.trial144.stderr:227451: waiting osd.1 seq 42949672993 2026-02-19T12:57:49.523 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:57:49.523 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672994 -lt 42949672993 2026-02-19T12:57:49.523 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:49.523 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509473 2026-02-19T12:57:49.523 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:49.523 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:57:49.523 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509473 2026-02-19T12:57:49.523 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:49.523 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509473 2026-02-19T12:57:49.523 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509473' 2026-02-19T12:57:49.523 INFO:tasks.workunit.client.0.trial144.stderr:227451: waiting osd.2 seq 64424509473 2026-02-19T12:57:49.523 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:57:49.523 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509473 -lt 64424509473 2026-02-19T12:57:49.534 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: fluubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836513 2026-02-19T12:57:49.534 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:49.534 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:57:49.534 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836513 2026-02-19T12:57:49.534 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:49.534 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836513 2026-02-19T12:57:49.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836513' 2026-02-19T12:57:49.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: waiting osd.0 seq 21474836513 2026-02-19T12:57:49.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:49.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836512 -lt 21474836513 2026-02-19T12:57:49.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:57:49.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:57:49.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:57:49.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836514 -lt 21474836513 2026-02-19T12:57:49.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:49.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672992 2026-02-19T12:57:49.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:49.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:57:49.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672992 2026-02-19T12:57:49.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:49.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672992 2026-02-19T12:57:49.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672992' 2026-02-19T12:57:49.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: waiting osd.1 seq 42949672992 2026-02-19T12:57:49.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:57:49.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672994 -lt 42949672992 2026-02-19T12:57:49.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:49.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509472 2026-02-19T12:57:49.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:49.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:57:49.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509472 2026-02-19T12:57:49.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:49.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509472 2026-02-19T12:57:49.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509472' 2026-02-19T12:57:49.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: waiting osd.2 seq 64424509472 2026-02-19T12:57:49.537 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:57:49.537 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509473 -lt 64424509472 2026-02-19T12:57:50.020 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-433791696899 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-433791696899 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=433791696899 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 433791696899' 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: waiting osd.3 seq 433791696899 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 433791696899 -lt 433791696899 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182432 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-107374182432 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182432 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 107374182432' 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: waiting osd.4 seq 107374182432 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182432 -lt 107374182432 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-433791696899 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-433791696899 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=433791696899 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 433791696899' 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: waiting osd.5 seq 433791696899 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 433791696899 -lt 433791696899 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-146028888094 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:50.021 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:57:50.038 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qsh_pg_stats: for s in $seqs 2026-02-19T12:57:50.038 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-433791696898 2026-02-19T12:57:50.038 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:50.038 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:57:50.038 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-433791696898 2026-02-19T12:57:50.038 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:50.038 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=433791696898 2026-02-19T12:57:50.038 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 433791696898' 2026-02-19T12:57:50.038 INFO:tasks.workunit.client.0.trial144.stderr:227448: waiting osd.3 seq 433791696898 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 433791696899 -lt 433791696898 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182431 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-107374182431 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182431 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 107374182431' 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: waiting osd.4 seq 107374182431 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182432 -lt 107374182431 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-433791696898 2026-02-19T12:57:50.039 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:50.040 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:57:50.040 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-433791696898 2026-02-19T12:57:50.040 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:50.040 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=433791696898 2026-02-19T12:57:50.040 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 433791696898' 2026-02-19T12:57:50.040 INFO:tasks.workunit.client.0.trial144.stderr:227448: waiting osd.5 seq 433791696898 2026-02-19T12:57:50.040 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:57:50.040 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 433791696899 -lt 433791696898 2026-02-19T12:57:50.040 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:50.040 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-146028888093 2026-02-19T12:57:50.040 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:50.040 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:57:50.514 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-146028888094 2026-02-19T12:57:50.514 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:50.514 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888094 2026-02-19T12:57:50.514 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 146028888094' 2026-02-19T12:57:50.515 INFO:tasks.workunit.client.0.trial144.stderr:227451: waiting osd.6 seq 146028888094 2026-02-19T12:57:50.515 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:57:50.515 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888094 -lt 146028888094 2026-02-19T12:57:50.515 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:50.515 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724573 2026-02-19T12:57:50.516 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:50.516 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:57:50.516 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-167503724573 2026-02-19T12:57:50.516 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:50.516 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724573 2026-02-19T12:57:50.516 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 167503724573' 2026-02-19T12:57:50.516 INFO:tasks.workunit.client.0.trial144.stderr:227451: waiting osd.7 seq 167503724573 2026-02-19T12:57:50.517 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:57:50.517 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724574 -lt 167503724573 2026-02-19T12:57:50.517 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:50.517 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593756 2026-02-19T12:57:50.517 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:50.517 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:57:50.518 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-184683593756 2026-02-19T12:57:50.518 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:50.518 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593756 2026-02-19T12:57:50.518 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 184683593756' 2026-02-19T12:57:50.518 INFO:tasks.workunit.client.0.trial144.stderr:227451: waiting osd.8 seq 184683593756 2026-02-19T12:57:50.518 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:57:50.518 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593757 -lt 184683593756 2026-02-19T12:57:50.519 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:50.519 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-412316860422 2026-02-19T12:57:50.519 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:50.519 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:57:50.519 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-412316860422 2026-02-19T12:57:50.519 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:50.519 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=412316860422 2026-02-19T12:57:50.552 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting a/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-146028888093 2026-02-19T12:57:50.552 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:50.552 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888093 2026-02-19T12:57:50.552 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 146028888093' 2026-02-19T12:57:50.552 INFO:tasks.workunit.client.0.trial144.stderr:227448: waiting osd.6 seq 146028888093 2026-02-19T12:57:50.552 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:57:50.552 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888094 -lt 146028888093 2026-02-19T12:57:50.552 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:50.552 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724572 2026-02-19T12:57:50.552 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-167503724572 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724572 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 167503724572' 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: waiting osd.7 seq 167503724572 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724574 -lt 167503724572 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593755 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-184683593755 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593755 2026-02-19T12:57:50.553 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 184683593755' 2026-02-19T12:57:50.554 INFO:tasks.workunit.client.0.trial144.stderr:227448: waiting osd.8 seq 184683593755 2026-02-19T12:57:50.554 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:57:50.554 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593757 -lt 184683593755 2026-02-19T12:57:50.554 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:57:50.554 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-412316860423 2026-02-19T12:57:50.554 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:57:50.554 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:57:50.554 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-412316860423 2026-02-19T12:57:50.554 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:57:50.554 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=412316860423 2026-02-19T12:57:51.480 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 412316860422' 2026-02-19T12:57:51.480 INFO:tasks.workunit.client.0.trial144.stderr:227451: waiting osd.9 seq 412316860422 2026-02-19T12:57:51.480 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:57:51.480 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 412316860423 -lt 412316860422 2026-02-19T12:57:51.480 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:57:51.480 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:51.481 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:51.481 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:57:51.481 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:51.481 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:51.481 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:51.481 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:51.481 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:51.481 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:51.481 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:51.481 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:57:51.481 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:51.481 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:51.482 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:51.482 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:57:51.482 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-19T12:57:51.482 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T12:57:51.482 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-19T12:57:51.482 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:57:51.482 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T12:57:51.482 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:57:51.482 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:51.482 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:51.482 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:51.482 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:51.483 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:51.483 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:51.483 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:51.533 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtosd.9 seq 412316860423' 2026-02-19T12:57:51.534 INFO:tasks.workunit.client.0.trial144.stderr:227448: waiting osd.9 seq 412316860423 2026-02-19T12:57:51.534 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:57:51.534 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 412316860423 -lt 412316860423 2026-02-19T12:57:51.534 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:57:51.534 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:51.534 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:51.534 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:57:51.534 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:51.534 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:51.534 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:51.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:51.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:51.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:51.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:51.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:57:51.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:51.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:51.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:51.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:57:51.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-19T12:57:51.535 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T12:57:51.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-19T12:57:51.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:57:51.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T12:57:51.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:57:51.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:51.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:51.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:51.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:51.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:51.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:51.536 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:52.443 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:52.444 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:52.444 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:52.444 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:52.444 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:52.444 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:57:52.444 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:52.444 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:52.444 INFO:tasks.workunit.client.0.trial144.stderr:227451: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:52.444 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:57:52.444 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:57:52.444 INFO:tasks.workunit.client.0.trial144.stderr:227451: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:57:52.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:57:52.464 INFO:tasks.workunit.client.0.trial144.stderr:est/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:57:52.464 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:52.464 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:52.464 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:52.464 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-19T12:57:52.464 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-19T12:57:52.464 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T12:57:52.464 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T12:57:52.464 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T12:57:52.464 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T12:57:52.465 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T12:57:52.465 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T12:57:52.465 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T12:57:52.465 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T12:57:52.465 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T12:57:52.465 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T12:57:52.465 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T12:57:52.465 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T12:57:52.465 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T12:57:52.465 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:57:52.465 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:57:52.466 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:57:52.466 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:57:52.466 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:57:52.466 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:57:52.466 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:57:52.466 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:57:52.466 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:57:52.466 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:57:52.466 INFO:tasks.workunit.client.0.trial144.stderr:227448: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:57:52.466 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:57:52.466 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:57:52.467 INFO:tasks.workunit.client.0.trial144.stderr:227448: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:57:52.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:57:52.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:57:52.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 227447 2026-02-19T12:57:52.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:57:52.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:57:52.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:57:52.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:257: corrupt_and_repair_two: return_code=0 2026-02-19T12:57:52.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:258: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:57:52.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-19T12:57:52.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T12:57:52.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:57:52.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T12:57:52.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:57:52.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_two: local pg=2.0 2026-02-19T12:57:52.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:264: corrupt_and_repair_two: repair 2.0 2026-02-19T12:57:52.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T12:57:52.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T12:57:52.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:52.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:52.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:52.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:52.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:57:28.446265+0000 2026-02-19T12:57:52.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T12:57:52.934 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-19T12:57:52.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T12:57:28.446265+0000 2026-02-19T12:57:52.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T12:57:52.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:57:28.446265+0000 2026-02-19T12:57:52.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:57:52.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:57:52.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:52.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:52.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:52.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:52.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:52.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:53.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:57:28.446265+0000 '>' 2026-02-19T12:57:28.446265+0000 2026-02-19T12:57:53.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:57:54.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:57:54.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:54.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:54.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:54.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:54.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:54.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:54.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:57:28.446265+0000 '>' 2026-02-19T12:57:28.446265+0000 2026-02-19T12:57:54.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:57:55.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:57:55.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:55.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:55.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:55.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:55.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:55.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:55.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:57:28.446265+0000 '>' 2026-02-19T12:57:28.446265+0000 2026-02-19T12:57:55.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:57:56.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:57:56.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:56.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:56.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:56.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:56.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:56.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:56.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:57:28.446265+0000 '>' 2026-02-19T12:57:28.446265+0000 2026-02-19T12:57:56.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:57:57.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:57:57.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:57.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:57.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:57.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:57.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:57.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:57.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:57:28.446265+0000 '>' 2026-02-19T12:57:28.446265+0000 2026-02-19T12:57:57.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:57:58.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:57:58.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:57:58.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T12:57:58.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T12:57:58.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:57:58.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:57:58.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T12:57:58.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:57:53.545826+0000 '>' 2026-02-19T12:57:28.446265+0000 2026-02-19T12:57:58.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:57:58.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:268: corrupt_and_repair_two: pids= 2026-02-19T12:57:58.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:269: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:57:58.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:57:58.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:57:58.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 231126"' 2026-02-19T12:57:58.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 231126' 2026-02-19T12:57:58.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:270: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:57:58.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T12:57:58.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T12:57:58.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 231128"' 2026-02-19T12:57:58.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:57:58.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 231128' 2026-02-19T12:57:58.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:271: corrupt_and_repair_two: wait_background pids 2026-02-19T12:57:58.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 231126 231128' 2026-02-19T12:57:58.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T12:57:58.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:57:58.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 231126 2026-02-19T12:57:58.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/231129: /' 2026-02-19T12:57:58.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:57:58.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/231132: /' 2026-02-19T12:57:59.402 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:57:59.402 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:57:59.402 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-19T12:57:59.403 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:57:59.403 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:57:59.403 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:57:59.403 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:57:59.403 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-19T12:57:59.403 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:57:59.403 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-19T12:57:59.403 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:57:59.403 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:57:59.403 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:57:59.403 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:57:59.403 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:57:59.404 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:57:59.404 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-19T12:57:59.404 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:57:59.404 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:57:59.404 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-19T12:57:59.404 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:57:59.404 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:57:59.404 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING list-attrs 2026-02-19T12:57:59.404 INFO:tasks.workunit.client.0.trial144.stderr:231132: _ 2026-02-19T12:57:59.404 INFO:tasks.workunit.client.0.trial144.stderr:231132: hinfo_key 2026-02-19T12:57:59.404 INFO:tasks.workunit.client.0.trial144.stderr:231132: snapset 2026-02-19T12:57:59.404 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-19T12:57:59.405 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:57:59.405 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:57:59.405 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-19T12:57:59.405 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:57:59.405 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-19T12:57:59.405 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:57:59.405 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:57:59.406 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:57:59.406 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:57:59.406 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-19T12:57:59.406 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-19T12:57:59.406 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:57:59.406 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:57:59.406 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:57:59.406 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:57:59.406 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:57:59.406 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:57:59.406 INFO:tasks.workunit.client.0.trial144.stderr:231132: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:57:59.407 INFO:tasks.workunit.client.0.trial144.stderr:231132: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:57:59.407 INFO:tasks.workunit.client.0.trial144.stderr:231132: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:57:59.407 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:59.407 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:59.407 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:57:59.407 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:57:59.407 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:57:59.407 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:57:59.407 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:57:59.407 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:57:59.408 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:57:59.408 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:57:59.408 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:57:59.408 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-19T12:57:59.408 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-19T12:57:59.408 INFO:tasks.workunit.client.0.trial144.stderr:231132: start osd.5 2026-02-19T12:57:59.423 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_reco231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:57:59.423 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:57:59.423 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T12:57:59.423 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:57:59.423 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:57:59.423 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:57:59.423 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:57:59.423 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T12:57:59.423 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:57:59.424 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T12:57:59.424 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:57:59.424 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:57:59.424 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:57:59.424 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:57:59.424 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:57:59.424 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:57:59.424 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-19T12:57:59.424 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:57:59.424 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:57:59.424 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T12:57:59.424 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:57:59.425 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:57:59.425 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-19T12:57:59.425 INFO:tasks.workunit.client.0.trial144.stderr:231129: Error getting attr on : 2.0s0_head,0#-4:00000000:::deep_scrub_2.0s0:head#, (61) No data available 2026-02-19T12:57:59.425 INFO:tasks.workunit.client.0.trial144.stderr:231129: Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-19T12:57:59.425 INFO:tasks.workunit.client.0.trial144.stderr:231129: _ 2026-02-19T12:57:59.425 INFO:tasks.workunit.client.0.trial144.stderr:231129: hinfo_key 2026-02-19T12:57:59.425 INFO:tasks.workunit.client.0.trial144.stderr:231129: snapset 2026-02-19T12:57:59.425 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T12:57:59.425 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:57:59.425 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:57:59.425 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T12:57:59.425 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:57:59.425 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T12:57:59.426 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:57:59.426 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:57:59.426 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:57:59.426 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:57:59.426 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T12:57:59.427 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T12:57:59.427 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:57:59.427 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:57:59.427 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:57:59.427 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:57:59.427 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:57:59.427 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:57:59.427 INFO:tasks.workunit.client.0.trial144.stderr:231129: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:57:59.427 INFO:tasks.workunit.client.0.trial144.stderr:231129: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:57:59.427 INFO:tasks.workunit.client.0.trial144.stderr:231129: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:57:59.427 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:59.427 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:57:59.428 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:57:59.428 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:57:59.428 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:57:59.428 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:57:59.428 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:57:59.428 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:57:59.428 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:57:59.428 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:57:59.428 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:57:59.428 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T12:57:59.428 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T12:57:59.428 INFO:tasks.workunit.client.0.trial144.stderr:231129: start osd.3 2026-02-19T12:58:02.038 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.very_ops 2026-02-19T12:58:02.038 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-19T12:58:02.038 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-19T12:58:02.038 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: 2026-02-19T12:57:59.423+0000 7f867e256a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: 2026-02-19T12:57:59.423+0000 7f867e256a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: 2026-02-19T12:57:59.423+0000 7f867e256a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: 0 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:02.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: 2026-02-19T12:57:59.891+0000 7f867e256a40 -1 Falling back to public interface 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: 1 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: 2026-02-19T12:58:00.819+0000 7f867e256a40 -1 osd.5 102 log_to_monitors true 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: 2 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: osd.5 up in weight 1 up_from 106 up_thru 106 down_at 103 last_clean_interval [101,102) [v2:127.0.0.1:6842/3060576502,v1:127.0.0.1:6843/3060576502] [v2:127.0.0.1:6844/3060576502,v1:127.0.0.1:6845/3060576502] exists,up f2b41730-2834-41c5-a57f-2a8a981fe2c0 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:58:02.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:58:02.041 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:58:02.071 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntulog' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:58:02.071 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T12:58:02.071 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T12:58:02.071 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:58:02.071 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:58:02.071 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:58:02.071 INFO:tasks.workunit.client.0.trial144.stderr:231129: 2026-02-19T12:57:59.439+0000 7fe008183a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:02.071 INFO:tasks.workunit.client.0.trial144.stderr:231129: 2026-02-19T12:57:59.443+0000 7fe008183a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:02.071 INFO:tasks.workunit.client.0.trial144.stderr:231129: 2026-02-19T12:57:59.443+0000 7fe008183a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:02.071 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T12:58:02.071 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:58:02.071 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T12:58:02.071 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: 0 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: 2026-02-19T12:57:59.875+0000 7fe008183a40 -1 Falling back to public interface 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: 1 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: 2026-02-19T12:58:00.823+0000 7fe008183a40 -1 osd.3 102 log_to_monitors true 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: 2 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: osd.3 up in weight 1 up_from 106 up_thru 106 down_at 103 last_clean_interval [101,102) [v2:127.0.0.1:6826/360921094,v1:127.0.0.1:6827/360921094] [v2:127.0.0.1:6828/360921094,v1:127.0.0.1:6829/360921094] exists,up ef3f1d22-1abd-4877-a9fc-b7ff790fe73e 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:58:02.072 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone./cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231132: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231132: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:58:02.528 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: 1 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: 2 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: 3 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: 4 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: 5 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: 6 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: 7 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: 8 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: 9' 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836518 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836518 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518' 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672997 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672997 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518 1-42949672997' 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509477 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509477 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-64424509477' 2026-02-19T12:58:02.529 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standaloclient.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:58:02.548 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: 1 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: 2 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: 3 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: 4 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: 5 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: 6 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: 7 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: 8 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: 9' 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836519 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836519 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836519' 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672998 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672998 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836519 1-42949672998' 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509478 2026-02-19T12:58:02.549 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509478 2026-02-19T12:58:02.978 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-214748365ne/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:58:02.978 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=455266533378 2026-02-19T12:58:02.978 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 455266533378 2026-02-19T12:58:02.978 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-64424509477 3-455266533378' 2026-02-19T12:58:02.978 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.978 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:58:02.978 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182436 2026-02-19T12:58:02.978 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182436 2026-02-19T12:58:02.978 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-64424509477 3-455266533378 4-107374182436' 2026-02-19T12:58:02.978 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.978 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:58:02.978 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=455266533378 2026-02-19T12:58:02.978 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 455266533378 2026-02-19T12:58:02.978 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-64424509477 3-455266533378 4-107374182436 5-455266533378' 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888098 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888098 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-64424509477 3-455266533378 4-107374182436 5-455266533378 6-146028888098' 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724577 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724577 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-64424509477 3-455266533378 4-107374182436 5-455266533378 6-146028888098 7-167503724577' 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593760 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593760 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-64424509477 3-455266533378 4-107374182436 5-455266533378 6-146028888098 7-167503724577 8-184683593760' 2026-02-19T12:58:02.979 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: 19 1-42949672998 2-64424509478' 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=455266533379 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 455266533379 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509478 3-455266533379' 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182437 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182437 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509478 3-455266533379 4-107374182437' 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=455266533379 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 455266533379 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509478 3-455266533379 4-107374182437 5-455266533379' 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T12:58:02.982 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888099 2026-02-19T12:58:02.983 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888099 2026-02-19T12:58:02.983 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509478 3-455266533379 4-107374182437 5-455266533379 6-146028888099' 2026-02-19T12:58:02.983 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.983 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T12:58:02.983 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724578 2026-02-19T12:58:02.983 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724578 2026-02-19T12:58:02.983 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509478 3-455266533379 4-107374182437 5-455266533379 6-146028888099 7-167503724578' 2026-02-19T12:58:02.983 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:02.983 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T12:58:02.983 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593761 2026-02-19T12:58:02.983 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593761 2026-02-19T12:58:04.546 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509478 3-455266533379 4-107374182437 5-455266533379 6-146028888099 7-167503724578 8-1846835flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:58:04.546 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=412316860427 2026-02-19T12:58:04.546 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 412316860427 2026-02-19T12:58:04.546 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518 1-42949672997 2-64424509477 3-455266533378 4-107374182436 5-455266533378 6-146028888098 7-167503724577 8-184683593760 9-412316860427' 2026-02-19T12:58:04.546 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:04.546 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836518 2026-02-19T12:58:04.547 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:04.547 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:58:04.547 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836518 2026-02-19T12:58:04.547 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:04.547 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836518 2026-02-19T12:58:04.547 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836518' 2026-02-19T12:58:04.547 INFO:tasks.workunit.client.0.trial144.stderr:231132: waiting osd.0 seq 21474836518 2026-02-19T12:58:04.547 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:58:04.547 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836517 -lt 21474836518 2026-02-19T12:58:04.547 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:58:04.548 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:58:04.548 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:58:04.548 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836520 -lt 21474836518 2026-02-19T12:58:04.548 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:04.548 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672997 2026-02-19T12:58:04.548 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:04.548 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:58:04.548 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672997 2026-02-19T12:58:04.548 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:04.548 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672997 2026-02-19T12:58:04.548 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672997' 2026-02-19T12:58:04.548 INFO:tasks.workunit.client.0.trial144.stderr:231132: waiting osd.1 seq 42949672997 2026-02-19T12:58:04.549 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:58:04.549 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672999 -lt 42949672997 2026-02-19T12:58:04.549 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:04.549 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509477 2026-02-19T12:58:04.549 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:04.549 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:58:04.549 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509477 2026-02-19T12:58:04.551 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:93761' 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=412316860428 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 412316860428 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836519 1-42949672998 2-64424509478 3-455266533379 4-107374182437 5-455266533379 6-146028888099 7-167503724578 8-184683593761 9-412316860428' 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836519 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836519 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836519 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836519' 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: waiting osd.0 seq 21474836519 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836517 -lt 21474836519 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836520 -lt 21474836519 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672998 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672998 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672998 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672998' 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: waiting osd.1 seq 42949672998 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672999 -lt 42949672998 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509478 2026-02-19T12:58:04.552 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:05.038 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:05.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509477 2026-02-19T12:58:05.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509477' 2026-02-19T12:58:05.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: waiting osd.2 seq 64424509477 2026-02-19T12:58:05.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:58:05.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509478 -lt 64424509477 2026-02-19T12:58:05.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:05.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-455266533378 2026-02-19T12:58:05.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:05.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:58:05.039 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-455266533378 2026-02-19T12:58:05.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:05.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=455266533378 2026-02-19T12:58:05.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 455266533378' 2026-02-19T12:58:05.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: waiting osd.3 seq 455266533378 2026-02-19T12:58:05.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:58:05.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 455266533379 -lt 455266533378 2026-02-19T12:58:05.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:05.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182436 2026-02-19T12:58:05.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:05.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:58:05.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-107374182436 2026-02-19T12:58:05.040 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:05.041 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182436 2026-02-19T12:58:05.041 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 107374182436' 2026-02-19T12:58:05.041 INFO:tasks.workunit.client.0.trial144.stderr:231132: waiting osd.4 seq 107374182436 2026-02-19T12:58:05.041 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:58:05.041 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182438 -lt 107374182436 2026-02-19T12:58:05.041 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:05.041 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-455266533378 2026-02-19T12:58:05.041 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:05.041 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:58:05.041 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-455266533378 2026-02-19T12:58:05.041 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:05.041 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=455266533378 2026-02-19T12:58:05.041 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 455266533378' 2026-02-19T12:58:05.042 INFO:tasks.workunit.client.0.trial144.stderr:231132: waiting osd.5 seq 455266533378 2026-02-19T12:58:05.054 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg: osd=2 2026-02-19T12:58:05.054 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509478 2026-02-19T12:58:05.054 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:05.054 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509478 2026-02-19T12:58:05.054 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509478' 2026-02-19T12:58:05.054 INFO:tasks.workunit.client.0.trial144.stderr:231129: waiting osd.2 seq 64424509478 2026-02-19T12:58:05.055 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:58:05.055 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509478 -lt 64424509478 2026-02-19T12:58:05.055 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:05.055 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-455266533379 2026-02-19T12:58:05.055 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:05.055 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T12:58:05.055 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-455266533379 2026-02-19T12:58:05.055 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:05.055 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=455266533379 2026-02-19T12:58:05.055 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 455266533379' 2026-02-19T12:58:05.055 INFO:tasks.workunit.client.0.trial144.stderr:231129: waiting osd.3 seq 455266533379 2026-02-19T12:58:05.055 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T12:58:05.055 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 455266533379 -lt 455266533379 2026-02-19T12:58:05.056 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:05.056 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182437 2026-02-19T12:58:05.056 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:05.056 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T12:58:05.056 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-107374182437 2026-02-19T12:58:05.056 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:05.056 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182437 2026-02-19T12:58:05.056 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 107374182437' 2026-02-19T12:58:05.056 INFO:tasks.workunit.client.0.trial144.stderr:231129: waiting osd.4 seq 107374182437 2026-02-19T12:58:05.056 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T12:58:05.056 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182438 -lt 107374182437 2026-02-19T12:58:05.056 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:05.056 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-455266533379 2026-02-19T12:58:05.057 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:05.057 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T12:58:05.057 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-455266533379 2026-02-19T12:58:05.057 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:05.057 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=455266533379 2026-02-19T12:58:05.696 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/cep_stats: ceph osd last-stat-seq 5 2026-02-19T12:58:05.696 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 455266533379 -lt 455266533378 2026-02-19T12:58:05.696 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:05.696 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-146028888098 2026-02-19T12:58:05.696 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:05.696 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:58:05.696 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-146028888098 2026-02-19T12:58:05.697 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:05.697 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888098 2026-02-19T12:58:05.697 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 146028888098' 2026-02-19T12:58:05.697 INFO:tasks.workunit.client.0.trial144.stderr:231132: waiting osd.6 seq 146028888098 2026-02-19T12:58:05.697 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:58:05.697 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888099 -lt 146028888098 2026-02-19T12:58:05.697 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:05.697 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724577 2026-02-19T12:58:05.697 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:05.697 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:58:05.697 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-167503724577 2026-02-19T12:58:05.697 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:05.698 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724577 2026-02-19T12:58:05.698 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 167503724577' 2026-02-19T12:58:05.698 INFO:tasks.workunit.client.0.trial144.stderr:231132: waiting osd.7 seq 167503724577 2026-02-19T12:58:05.698 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:58:05.698 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724578 -lt 167503724577 2026-02-19T12:58:05.698 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:05.698 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593760 2026-02-19T12:58:05.698 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:05.698 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:58:05.698 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-184683593760 2026-02-19T12:58:05.698 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:05.698 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593760 2026-02-19T12:58:05.698 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 184683593760' 2026-02-19T12:58:05.699 INFO:tasks.workunit.client.0.trial144.stderr:231132: waiting osd.8 seq 184683593760 2026-02-19T12:58:05.699 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:58:05.699 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593762 -lt 184683593760 2026-02-19T12:58:05.699 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:05.699 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-412316860427 2026-02-19T12:58:05.718 INFO:tasks.workunit.client.0.trial144.stderr:231132: //homh-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 455266533379' 2026-02-19T12:58:05.718 INFO:tasks.workunit.client.0.trial144.stderr:231129: waiting osd.5 seq 455266533379 2026-02-19T12:58:05.719 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T12:58:05.719 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 455266533379 -lt 455266533379 2026-02-19T12:58:05.719 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:05.719 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-146028888099 2026-02-19T12:58:05.719 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:05.719 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T12:58:05.719 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-146028888099 2026-02-19T12:58:05.719 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:05.719 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888099 2026-02-19T12:58:05.719 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 146028888099' 2026-02-19T12:58:05.719 INFO:tasks.workunit.client.0.trial144.stderr:231129: waiting osd.6 seq 146028888099 2026-02-19T12:58:05.719 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T12:58:05.720 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888099 -lt 146028888099 2026-02-19T12:58:05.720 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:05.720 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724578 2026-02-19T12:58:05.720 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:05.720 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T12:58:05.720 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-167503724578 2026-02-19T12:58:05.720 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:05.720 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724578 2026-02-19T12:58:05.720 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 167503724578' 2026-02-19T12:58:05.720 INFO:tasks.workunit.client.0.trial144.stderr:231129: waiting osd.7 seq 167503724578 2026-02-19T12:58:05.720 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T12:58:05.720 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724578 -lt 167503724578 2026-02-19T12:58:05.720 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:05.721 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593761 2026-02-19T12:58:05.721 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:05.721 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T12:58:05.721 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-184683593761 2026-02-19T12:58:05.721 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:05.721 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593761 2026-02-19T12:58:05.721 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 184683593761' 2026-02-19T12:58:05.721 INFO:tasks.workunit.client.0.trial144.stderr:231129: waiting osd.8 seq 184683593761 2026-02-19T12:58:05.721 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T12:58:05.721 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593762 -lt 184683593761 2026-02-19T12:58:06.416 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.cliente/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:06.416 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:58:06.416 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-412316860427 2026-02-19T12:58:06.416 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:06.416 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=412316860427 2026-02-19T12:58:06.416 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 412316860427' 2026-02-19T12:58:06.416 INFO:tasks.workunit.client.0.trial144.stderr:231132: waiting osd.9 seq 412316860427 2026-02-19T12:58:06.417 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:58:06.417 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 412316860428 -lt 412316860427 2026-02-19T12:58:06.417 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:58:06.417 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:58:06.417 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:58:06.417 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:58:06.417 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:58:06.417 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:58:06.417 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:58:06.417 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:58:06.417 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:58:06.418 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:58:06.418 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:58:06.418 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:58:06.418 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:58:06.418 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:58:06.418 INFO:tasks.workunit.client.0.trial144.stderr:231132: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:58:06.418 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:58:06.418 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:58:06.418 INFO:tasks.workunit.client.0.trial144.stderr:231132: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:58:06.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:58:06.443 INFO:tasks.workunit.client.0.trial144.stderr:.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:06.443 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-412316860428 2026-02-19T12:58:06.443 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:06.443 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T12:58:06.443 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-412316860428 2026-02-19T12:58:06.443 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=412316860428 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 412316860428' 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: waiting osd.9 seq 412316860428 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 412316860428 -lt 412316860428 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:58:06.444 INFO:tasks.workunit.client.0.trial144.stderr:231129: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:58:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T12:58:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T12:58:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 231128 2026-02-19T12:58:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T12:58:06.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T12:58:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T12:58:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:272: corrupt_and_repair_two: return_code=0 2026-02-19T12:58:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:273: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-19T12:58:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:275: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:58:06.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:276: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:58:06.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T12:58:06.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:58:06.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:58:06.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:58:06.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:58:06.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:58:06.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:58:06.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:58:06.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:58:06.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:58:06.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:58:06.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:58:06.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:58:06.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:58:06.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:58:06.584 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:58:06.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:58:06.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:58:06.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:58:06.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:58:06.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:58:06.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:58:06.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:58:06.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:58:06.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:06.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:06.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:58:06.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:58:06.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:58:06.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T12:58:06.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T12:58:06.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T12:58:06.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T12:58:06.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:58:06.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:58:06.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:58:06.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:58:06.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:58:06.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:58:06.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:58:06.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:58:06.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:58:06.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:58:06.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:58:06.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:58:06.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:58:06.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:58:06.635 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:58:06.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:58:06.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:58:06.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:58:06.636 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:58:06.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:58:06.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:58:06.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:58:06.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:58:06.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:06.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:06.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:58:06.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:58:06.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:58:06.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T12:58:06.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:58:06.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:06.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:06.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T12:58:06.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:58:06.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T12:58:06.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:58:06.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T12:58:06.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_and_repair_replicated td/osd-scrub-repair 2026-02-19T12:58:06.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:94: TEST_corrupt_and_repair_replicated: local dir=td/osd-scrub-repair 2026-02-19T12:58:06.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:95: TEST_corrupt_and_repair_replicated: local poolname=rbd 2026-02-19T12:58:06.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:97: TEST_corrupt_and_repair_replicated: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-19T12:58:06.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T12:58:06.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T12:58:06.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T12:58:06.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T12:58:06.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T12:58:06.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-19T12:58:06.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T12:58:06.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:58:06.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:58:06.670 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:58:06.670 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:06.670 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:06.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:06.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-19T12:58:06.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T12:58:06.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T12:58:06.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:58:06.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:58:06.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T12:58:06.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T12:58:06.699 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:58:06.699 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:58:06.699 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:58:06.700 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:58:06.700 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:06.700 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:06.700 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:58:06.700 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:58:06.701 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T12:58:06.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T12:58:06.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:58:06.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:58:06.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T12:58:06.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T12:58:06.759 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:58:06.759 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:58:06.759 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:58:06.759 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:58:06.759 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:06.759 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:06.759 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:58:06.759 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:58:06.759 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T12:58:06.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:98: TEST_corrupt_and_repair_replicated: run_mgr td/osd-scrub-repair x 2026-02-19T12:58:06.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T12:58:06.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T12:58:06.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T12:58:06.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T12:58:06.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T12:58:06.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T12:58:06.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T12:58:06.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:58:06.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:58:06.909 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:58:06.909 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:06.909 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:06.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:06.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:58:06.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:58:06.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:99: TEST_corrupt_and_repair_replicated: run_osd td/osd-scrub-repair 0 2026-02-19T12:58:06.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:58:06.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:58:06.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T12:58:06.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:58:06.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:58:06.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:58:06.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:58:06.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:58:06.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:58:06.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:58:06.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:58:06.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:58:06.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:58:06.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:58:06.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:58:06.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:58:06.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:58:06.933 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:58:06.933 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:06.933 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:06.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:06.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:06.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:58:06.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:58:06.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:58:06.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:58:06.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:58:06.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:58:06.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:58:06.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:58:06.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:58:06.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:58:06.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:58:06.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:58:06.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:58:06.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dadfb348-a305-4626-ab0d-a46db06d8dae 2026-02-19T12:58:06.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 dadfb348-a305-4626-ab0d-a46db06d8dae' 2026-02-19T12:58:06.936 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 dadfb348-a305-4626-ab0d-a46db06d8dae 2026-02-19T12:58:06.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:58:06.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDeCJdpa5p6OBAA4SGBRQAzGDeVgW3vxMRo3w== 2026-02-19T12:58:06.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDeCJdpa5p6OBAA4SGBRQAzGDeVgW3vxMRo3w=="}' 2026-02-19T12:58:06.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dadfb348-a305-4626-ab0d-a46db06d8dae -i td/osd-scrub-repair/0/new.json 2026-02-19T12:58:07.037 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:58:07.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T12:58:07.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDeCJdpa5p6OBAA4SGBRQAzGDeVgW3vxMRo3w== --osd-uuid dadfb348-a305-4626-ab0d-a46db06d8dae 2026-02-19T12:58:07.061 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:07.059+0000 7f9572651a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:07.063 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:07.063+0000 7f9572651a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:07.063 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:07.063+0000 7f9572651a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:07.064 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:07.063+0000 7f9572651a40 -1 bdev(0x558eb5796c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:58:07.064 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:07.063+0000 7f9572651a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T12:58:07.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T12:58:07.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:58:07.838 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:58:07.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T12:58:07.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:58:08.030 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:58:08.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T12:58:08.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:58:08.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:58:08.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:58:08.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:58:08.046 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:08.043+0000 7fc1aac49a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:08.047 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:08.047+0000 7fc1aac49a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:08.049 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:08.047+0000 7fc1aac49a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:08.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T12:58:08.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:58:08.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:58:08.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:58:08.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:58:08.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:08.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:58:08.189 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:58:08.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:08.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:58:08.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:08.428 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:08.427+0000 7fc1aac49a40 -1 Falling back to public interface 2026-02-19T12:58:09.032 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:09.031+0000 7fc1aac49a40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:58:09.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:09.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:09.350 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:58:09.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:58:09.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:09.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:58:09.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:10.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:10.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:10.516 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:58:10.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:58:10.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:10.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:58:10.680 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2082055206,v1:127.0.0.1:6803/2082055206] [v2:127.0.0.1:6804/2082055206,v1:127.0.0.1:6805/2082055206] exists,up dadfb348-a305-4626-ab0d-a46db06d8dae 2026-02-19T12:58:10.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:58:10.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:58:10.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:58:10.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:100: TEST_corrupt_and_repair_replicated: run_osd td/osd-scrub-repair 1 2026-02-19T12:58:10.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:58:10.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:58:10.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T12:58:10.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:58:10.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:58:10.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:58:10.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:58:10.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:58:10.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:58:10.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:58:10.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:58:10.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:58:10.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:58:10.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:58:10.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:58:10.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:58:10.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:58:10.685 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:58:10.685 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:10.685 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:10.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:10.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:10.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:58:10.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:58:10.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:58:10.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:58:10.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:58:10.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:58:10.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:58:10.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:58:10.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:58:10.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:58:10.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:58:10.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:58:10.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:58:10.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a3608c17-20f6-4aba-9382-efddb645ce47 2026-02-19T12:58:10.687 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 a3608c17-20f6-4aba-9382-efddb645ce47 2026-02-19T12:58:10.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 a3608c17-20f6-4aba-9382-efddb645ce47' 2026-02-19T12:58:10.688 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:58:10.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDiCJdpMK6ZKRAAphsk8KcqCFDG31KGL4Dw8g== 2026-02-19T12:58:10.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDiCJdpMK6ZKRAAphsk8KcqCFDG31KGL4Dw8g=="}' 2026-02-19T12:58:10.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a3608c17-20f6-4aba-9382-efddb645ce47 -i td/osd-scrub-repair/1/new.json 2026-02-19T12:58:10.861 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:58:10.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T12:58:10.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDiCJdpMK6ZKRAAphsk8KcqCFDG31KGL4Dw8g== --osd-uuid a3608c17-20f6-4aba-9382-efddb645ce47 2026-02-19T12:58:10.889 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:10.887+0000 7fdd6244fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:10.890 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:10.891+0000 7fdd6244fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:10.891 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:10.891+0000 7fdd6244fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:10.892 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:10.891+0000 7fdd6244fa40 -1 bdev(0x55836f602c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:58:10.892 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:10.891+0000 7fdd6244fa40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T12:58:11.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T12:58:11.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:58:11.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T12:58:11.527 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:58:11.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:58:11.731 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:58:11.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T12:58:11.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:58:11.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:58:11.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:58:11.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:58:11.748 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:11.747+0000 7f7ce57a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:11.749 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:11.747+0000 7f7ce57a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:11.750 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:11.747+0000 7f7ce57a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:11.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T12:58:11.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:58:11.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:58:11.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:58:11.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:58:11.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:11.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:58:11.888 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:58:11.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:11.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:58:12.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:12.128 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:12.127+0000 7f7ce57a4a40 -1 Falling back to public interface 2026-02-19T12:58:12.721 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:12.719+0000 7f7ce57a4a40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:58:13.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:13.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:13.047 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:58:13.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:58:13.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:13.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:58:13.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:14.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:14.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:14.208 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:58:14.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:58:14.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:14.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:58:14.368 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/126792511,v1:127.0.0.1:6811/126792511] [v2:127.0.0.1:6812/126792511,v1:127.0.0.1:6813/126792511] exists,up a3608c17-20f6-4aba-9382-efddb645ce47 2026-02-19T12:58:14.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:58:14.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:58:14.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:58:14.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:101: TEST_corrupt_and_repair_replicated: create_rbd_pool 2026-02-19T12:58:14.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T12:58:14.518 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T12:58:14.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T12:58:14.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T12:58:14.740 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T12:58:14.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:58:15.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T12:58:16.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:102: TEST_corrupt_and_repair_replicated: wait_for_clean 2026-02-19T12:58:16.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:58:16.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:58:16.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:58:16.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:58:16.033 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:58:16.033 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:58:16.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:58:16.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:58:16.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:58:16.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:58:16.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:58:16.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:58:16.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:58:16.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:58:16.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:58:16.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:58:16.307 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T12:58:16.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:58:16.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:16.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:58:16.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T12:58:16.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T12:58:16.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T12:58:16.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:16.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:58:16.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705666 2026-02-19T12:58:16.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705666 2026-02-19T12:58:16.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705666' 2026-02-19T12:58:16.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:16.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T12:58:16.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:16.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:58:16.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T12:58:16.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:16.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T12:58:16.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T12:58:16.455 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T12:58:16.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:58:16.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T12:58:16.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:58:17.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:58:17.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:58:17.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-19T12:58:17.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:17.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705666 2026-02-19T12:58:17.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:17.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:58:17.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705666 2026-02-19T12:58:17.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:17.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705666 2026-02-19T12:58:17.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705666' 2026-02-19T12:58:17.782 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705666 2026-02-19T12:58:17.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:58:17.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705666 -lt 38654705666 2026-02-19T12:58:17.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:58:17.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:58:17.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:58:18.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:58:18.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:58:18.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:58:18.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:58:18.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:58:18.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:58:18.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:58:18.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:58:18.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:58:18.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:58:18.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:58:18.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:58:18.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:58:18.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:58:18.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:58:18.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:104: TEST_corrupt_and_repair_replicated: add_something td/osd-scrub-repair rbd 2026-02-19T12:58:18.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T12:58:18.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=rbd 2026-02-19T12:58:18.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-19T12:58:18.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T12:58:18.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T12:58:18.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T12:58:18.705 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T12:58:18.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T12:58:18.910 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T12:58:18.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T12:58:18.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T12:58:18.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool rbd put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-19T12:58:18.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:105: TEST_corrupt_and_repair_replicated: get_not_primary rbd SOMETHING 2026-02-19T12:58:18.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=rbd 2026-02-19T12:58:18.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T12:58:18.948 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary rbd SOMETHING 2026-02-19T12:58:18.948 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-19T12:58:18.949 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:58:18.949 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd SOMETHING 2026-02-19T12:58:18.949 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:58:19.107 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T12:58:19.107 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map rbd SOMETHING 2026-02-19T12:58:19.107 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T12:58:19.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:105: TEST_corrupt_and_repair_replicated: corrupt_and_repair_one td/osd-scrub-repair rbd 0 2026-02-19T12:58:19.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:286: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-19T12:58:19.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:287: corrupt_and_repair_one: local poolname=rbd 2026-02-19T12:58:19.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:288: corrupt_and_repair_one: local osd=0 2026-02-19T12:58:19.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:293: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T12:58:19.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:58:19.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:58:19.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:58:19.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:58:19.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T12:58:19.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:58:19.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:58:19.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:58:19.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:58:19.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:58:19.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:58:19.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:58:19.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:58:19.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:58:19.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:58:19.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:58:19.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T12:58:19.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:58:19.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:58:19.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:58:19.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:58:19.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:58:19.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-19T12:58:19.763 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-19T12:58:19.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T12:58:19.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:58:19.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:58:19.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:58:19.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:58:19.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:58:19.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:58:19.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:58:19.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:58:19.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:58:19.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:58:19.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:58:19.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:58:19.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:58:19.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:58:19.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:58:19.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:58:19.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:58:19.870 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:58:19.871 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:19.871 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:19.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:19.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:19.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:58:19.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:58:19.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:58:19.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:58:19.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:58:19.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:58:19.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:58:19.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:58:19.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:58:19.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:58:19.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:58:19.874 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:58:19.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:58:19.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:58:19.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:58:19.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:58:19.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:58:19.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:58:19.890 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:19.887+0000 7f100389aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:19.891 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:19.891+0000 7f100389aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:19.892 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:19.891+0000 7f100389aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:20.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:58:20.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:58:20.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:58:20.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:58:20.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:58:20.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:20.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:58:20.027 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:58:20.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:20.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:58:20.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:20.300 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:20.299+0000 7f100389aa40 -1 Falling back to public interface 2026-02-19T12:58:20.924 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:20.923+0000 7f100389aa40 -1 osd.0 18 log_to_monitors true 2026-02-19T12:58:21.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:21.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:21.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:58:21.191 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:58:21.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:21.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:58:21.346 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 22 up_thru 22 down_at 19 last_clean_interval [5,18) [v2:127.0.0.1:6802/3346194472,v1:127.0.0.1:6803/3346194472] [v2:127.0.0.1:6804/3346194472,v1:127.0.0.1:6805/3346194472] exists,up dadfb348-a305-4626-ab0d-a46db06d8dae 2026-02-19T12:58:21.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:58:21.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:58:21.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:58:21.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:58:21.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:58:21.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:58:21.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:58:21.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:58:21.348 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:58:21.349 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:58:21.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:58:21.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:58:21.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:58:21.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:58:21.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:58:21.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:58:21.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:58:21.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:58:21.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:58:21.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:58:21.621 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T12:58:21.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:58:21.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:21.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:58:21.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280514 2026-02-19T12:58:21.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280514 2026-02-19T12:58:21.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-94489280514' 2026-02-19T12:58:21.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:21.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:58:21.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705668 2026-02-19T12:58:21.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705668 2026-02-19T12:58:21.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-94489280514 1-38654705668' 2026-02-19T12:58:21.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:21.768 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-94489280514 2026-02-19T12:58:21.768 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:21.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:58:21.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-94489280514 2026-02-19T12:58:21.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:21.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280514 2026-02-19T12:58:21.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 94489280514' 2026-02-19T12:58:21.771 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 94489280514 2026-02-19T12:58:21.771 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:58:21.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 94489280514 2026-02-19T12:58:21.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:58:22.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:58:22.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:58:23.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 94489280514 2026-02-19T12:58:23.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:58:24.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:58:24.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:58:24.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280514 -lt 94489280514 2026-02-19T12:58:24.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:24.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705668 2026-02-19T12:58:24.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:24.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:58:24.255 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705668 2026-02-19T12:58:24.255 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:24.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705668 2026-02-19T12:58:24.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-19T12:58:24.257 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 38654705668 2026-02-19T12:58:24.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:58:24.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-19T12:58:24.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:58:24.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:58:24.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:58:24.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:58:24.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:58:24.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:58:24.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:58:24.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:58:24.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:58:24.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:58:24.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:58:24.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:58:24.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:58:24.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:58:24.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:58:24.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:58:24.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:58:24.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:58:24.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: get_pg rbd SOMETHING 2026-02-19T12:58:24.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=rbd 2026-02-19T12:58:24.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:58:24.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map rbd SOMETHING 2026-02-19T12:58:24.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:58:25.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: local pg=1.3 2026-02-19T12:58:25.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: corrupt_and_repair_one: repair 1.3 2026-02-19T12:58:25.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=1.3 2026-02-19T12:58:25.136 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 1.3 2026-02-19T12:58:25.136 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:58:25.136 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:58:25.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:58:25.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:58:25.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:58:14.741046+0000 2026-02-19T12:58:25.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 1.3 2026-02-19T12:58:25.443 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.3 on osd.1 to repair 2026-02-19T12:58:25.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 1.3 2026-02-19T12:58:14.741046+0000 2026-02-19T12:58:25.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.3 2026-02-19T12:58:25.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:58:14.741046+0000 2026-02-19T12:58:25.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:58:25.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:58:25.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:58:25.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T12:58:25.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:58:25.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:58:25.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:58:25.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:58:25.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:58:14.741046+0000 '>' 2026-02-19T12:58:14.741046+0000 2026-02-19T12:58:25.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:58:26.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:58:26.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:58:26.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T12:58:26.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:58:26.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:58:26.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:58:26.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:58:26.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:58:14.741046+0000 '>' 2026-02-19T12:58:14.741046+0000 2026-02-19T12:58:26.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:58:27.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:58:27.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:58:27.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T12:58:27.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:58:27.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:58:27.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:58:27.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:58:27.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:58:14.741046+0000 '>' 2026-02-19T12:58:14.741046+0000 2026-02-19T12:58:27.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:58:28.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:58:28.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:58:28.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T12:58:28.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:58:28.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:58:28.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:58:28.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:58:29.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:58:14.741046+0000 '>' 2026-02-19T12:58:14.741046+0000 2026-02-19T12:58:29.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:58:30.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:58:30.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:58:30.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T12:58:30.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:58:30.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:58:30.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:58:30.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:58:30.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:58:25.950682+0000 '>' 2026-02-19T12:58:14.741046+0000 2026-02-19T12:58:30.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:58:30.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:302: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:58:30.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:58:30.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:58:30.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:58:30.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:58:30.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:58:30.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:58:30.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:58:30.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:58:30.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:58:30.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:58:30.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:58:30.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:58:30.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:58:30.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:58:30.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:58:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:58:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:58:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:58:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:58:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:58:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:58:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:58:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-19T12:58:30.649 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:58:30.649 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:58:30.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:58:30.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:58:30.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:58:30.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:58:30.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:58:30.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:58:30.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:58:30.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:58:30.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:58:30.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:58:30.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:58:30.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:58:30.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:58:30.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:58:30.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:58:30.775 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:58:30.775 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:58:30.775 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:58:30.775 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:58:30.775 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:30.775 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:30.775 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:30.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:30.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:58:30.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:58:30.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:58:30.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:58:30.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:58:30.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:58:30.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:58:30.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:58:30.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:58:30.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:58:30.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:58:30.776 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:58:30.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:58:30.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:58:30.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:58:30.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:58:30.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:58:30.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:58:30.791 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:30.791+0000 7f3039c55a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:30.793 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:30.791+0000 7f3039c55a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:30.793 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:30.791+0000 7f3039c55a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:30.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:58:30.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:58:30.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:58:30.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:58:30.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:58:30.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:30.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:58:30.937 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:58:30.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:30.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:58:31.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:31.172 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:31.170+0000 7f3039c55a40 -1 Falling back to public interface 2026-02-19T12:58:31.820 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:31.818+0000 7f3039c55a40 -1 osd.1 27 log_to_monitors true 2026-02-19T12:58:32.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:32.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:32.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:58:32.101 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:58:32.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:32.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:58:32.256 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 30 up_thru 30 down_at 27 last_clean_interval [9,27) [v2:127.0.0.1:6810/4211815447,v1:127.0.0.1:6811/4211815447] [v2:127.0.0.1:6812/4211815447,v1:127.0.0.1:6813/4211815447] exists,up a3608c17-20f6-4aba-9382-efddb645ce47 2026-02-19T12:58:32.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:58:32.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:58:32.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:58:32.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:58:32.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:58:32.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:58:32.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:58:32.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:58:32.258 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:58:32.258 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:58:32.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:58:32.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:58:32.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:58:32.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:58:32.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:58:32.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:58:32.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:58:32.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:58:32.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:58:32.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:58:32.530 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T12:58:32.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:58:32.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:32.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:58:32.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280517 2026-02-19T12:58:32.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280517 2026-02-19T12:58:32.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-94489280517' 2026-02-19T12:58:32.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:32.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:58:32.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018882 2026-02-19T12:58:32.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018882 2026-02-19T12:58:32.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-94489280517 1-128849018882' 2026-02-19T12:58:32.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:32.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-94489280517 2026-02-19T12:58:32.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:32.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:58:32.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-94489280517 2026-02-19T12:58:32.677 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:32.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280517 2026-02-19T12:58:32.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 94489280517' 2026-02-19T12:58:32.678 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 94489280517 2026-02-19T12:58:32.679 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:58:32.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280516 -lt 94489280517 2026-02-19T12:58:32.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:58:33.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:58:33.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:58:34.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280517 -lt 94489280517 2026-02-19T12:58:34.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:34.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-128849018882 2026-02-19T12:58:34.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:34.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:58:34.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-128849018882 2026-02-19T12:58:34.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:34.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018882 2026-02-19T12:58:34.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 128849018882' 2026-02-19T12:58:34.017 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 128849018882 2026-02-19T12:58:34.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:58:34.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018882 -lt 128849018882 2026-02-19T12:58:34.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:58:34.182 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:58:34.182 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:58:34.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:58:34.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:58:34.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:58:34.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:58:34.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:58:34.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:58:34.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:58:34.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:58:34.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:58:34.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:58:34.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:58:34.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:58:34.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:58:34.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:58:34.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:58:34.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: corrupt_and_repair_one: rados --pool rbd get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:58:34.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:304: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:58:34.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:107: TEST_corrupt_and_repair_replicated: get_primary rbd SOMETHING 2026-02-19T12:58:34.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-19T12:58:34.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T12:58:34.744 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd SOMETHING 2026-02-19T12:58:34.744 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T12:58:34.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:107: TEST_corrupt_and_repair_replicated: corrupt_and_repair_one td/osd-scrub-repair rbd 1 2026-02-19T12:58:34.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:286: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-19T12:58:34.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:287: corrupt_and_repair_one: local poolname=rbd 2026-02-19T12:58:34.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:288: corrupt_and_repair_one: local osd=1 2026-02-19T12:58:34.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:293: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:58:34.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:58:34.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:58:34.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:58:34.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:58:34.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:58:34.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:58:34.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:58:34.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:58:34.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:58:34.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:58:34.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:58:34.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:58:34.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:58:34.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:58:34.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:58:35.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:58:35.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-19T12:58:35.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:58:35.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:58:35.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:58:35.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:58:35.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:58:35.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-19T12:58:35.403 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-19T12:58:35.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:58:35.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:58:35.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:58:35.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:58:35.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:58:35.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:58:35.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:58:35.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:58:35.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:58:35.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:58:35.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:58:35.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:58:35.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:58:35.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:58:35.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:58:35.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:58:35.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:58:35.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:58:35.498 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:58:35.498 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:35.498 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:35.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:58:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:58:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:58:35.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:58:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:58:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:58:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:58:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:58:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:58:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:58:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:58:35.501 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:58:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:58:35.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:58:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:58:35.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:58:35.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:58:35.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:58:35.519 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:35.518+0000 7f68308e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:35.520 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:35.518+0000 7f68308e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:35.521 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:35.518+0000 7f68308e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:35.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:58:35.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:58:35.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:58:35.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:58:35.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:58:35.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:35.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:58:35.659 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:58:35.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:35.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:58:35.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:35.864 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:35.862+0000 7f68308e0a40 -1 Falling back to public interface 2026-02-19T12:58:36.426 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:36.422+0000 7f68308e0a40 -1 osd.1 31 log_to_monitors true 2026-02-19T12:58:36.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:36.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:36.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:58:36.819 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:58:36.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:36.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:58:36.975 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 35 up_thru 35 down_at 32 last_clean_interval [30,31) [v2:127.0.0.1:6810/4220590113,v1:127.0.0.1:6811/4220590113] [v2:127.0.0.1:6812/4220590113,v1:127.0.0.1:6813/4220590113] exists,up a3608c17-20f6-4aba-9382-efddb645ce47 2026-02-19T12:58:36.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:58:36.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:58:36.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:58:36.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:58:36.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:58:36.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:58:36.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:58:36.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:58:36.977 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:58:36.977 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:58:36.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:58:36.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:58:36.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:58:37.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:58:37.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:58:37.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:58:37.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:58:37.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:58:37.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:58:37.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:58:37.256 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T12:58:37.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:58:37.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:37.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:58:37.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280519 2026-02-19T12:58:37.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280519 2026-02-19T12:58:37.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-94489280519' 2026-02-19T12:58:37.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:37.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:58:37.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855362 2026-02-19T12:58:37.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855362 2026-02-19T12:58:37.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-94489280519 1-150323855362' 2026-02-19T12:58:37.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:37.406 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-94489280519 2026-02-19T12:58:37.406 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:37.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:58:37.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-94489280519 2026-02-19T12:58:37.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:37.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280519 2026-02-19T12:58:37.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 94489280519' 2026-02-19T12:58:37.410 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 94489280519 2026-02-19T12:58:37.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:58:37.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280519 -lt 94489280519 2026-02-19T12:58:37.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:37.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-150323855362 2026-02-19T12:58:37.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:37.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:58:37.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-150323855362 2026-02-19T12:58:37.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:37.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855362 2026-02-19T12:58:37.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 150323855362' 2026-02-19T12:58:37.574 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 150323855362 2026-02-19T12:58:37.575 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:58:37.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855362 -lt 150323855362 2026-02-19T12:58:37.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:58:37.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:58:37.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:58:37.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:58:37.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:58:37.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:58:37.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:58:37.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:58:37.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:58:37.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:58:37.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:58:38.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:58:38.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:58:38.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:58:38.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:58:38.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:58:38.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:58:38.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:58:38.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: get_pg rbd SOMETHING 2026-02-19T12:58:38.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=rbd 2026-02-19T12:58:38.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T12:58:38.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map rbd SOMETHING 2026-02-19T12:58:38.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T12:58:38.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:297: corrupt_and_repair_one: local pg=1.3 2026-02-19T12:58:38.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: corrupt_and_repair_one: repair 1.3 2026-02-19T12:58:38.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=1.3 2026-02-19T12:58:38.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 1.3 2026-02-19T12:58:38.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:58:38.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:58:38.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:58:38.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:58:38.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T12:58:25.950682+0000 2026-02-19T12:58:38.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 1.3 2026-02-19T12:58:38.765 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.3 on osd.1 to repair 2026-02-19T12:58:38.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 1.3 2026-02-19T12:58:25.950682+0000 2026-02-19T12:58:38.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.3 2026-02-19T12:58:38.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:58:25.950682+0000 2026-02-19T12:58:38.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T12:58:38.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T12:58:38.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:58:38.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T12:58:38.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:58:38.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:58:38.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:58:38.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:58:38.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:58:25.950682+0000 '>' 2026-02-19T12:58:25.950682+0000 2026-02-19T12:58:38.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:58:39.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:58:39.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:58:39.937 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T12:58:39.937 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:58:39.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:58:39.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:58:39.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:58:40.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:58:25.950682+0000 '>' 2026-02-19T12:58:25.950682+0000 2026-02-19T12:58:40.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:58:41.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:58:41.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:58:41.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T12:58:41.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:58:41.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:58:41.100 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:58:41.100 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:58:41.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:58:25.950682+0000 '>' 2026-02-19T12:58:25.950682+0000 2026-02-19T12:58:41.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:58:42.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:58:42.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:58:42.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T12:58:42.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:58:42.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:58:42.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:58:42.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:58:42.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:58:25.950682+0000 '>' 2026-02-19T12:58:25.950682+0000 2026-02-19T12:58:42.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T12:58:43.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T12:58:43.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T12:58:43.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T12:58:43.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T12:58:43.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T12:58:43.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T12:58:43.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T12:58:43.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:58:39.405738+0000 '>' 2026-02-19T12:58:25.950682+0000 2026-02-19T12:58:43.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T12:58:43.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:302: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:58:43.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:58:43.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:58:43.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:58:43.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:58:43.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:58:43.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:58:43.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:58:43.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:58:43.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:58:43.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:58:43.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:58:43.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:58:43.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:58:43.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:58:43.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:58:43.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:58:43.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-19T12:58:43.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:58:43.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:58:43.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:58:43.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:58:43.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:58:43.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-19T12:58:43.952 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::deep_scrub_1.3:head#, (61) No data available 2026-02-19T12:58:43.953 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::scrub_1.3:head#, (61) No data available 2026-02-19T12:58:43.953 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T12:58:43.953 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T12:58:44.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:58:44.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:58:44.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:58:44.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:58:44.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:58:44.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:58:44.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:58:44.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:58:44.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:58:44.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:58:44.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:58:44.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:58:44.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:58:44.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:58:44.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:58:44.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:58:44.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:58:44.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:58:44.041 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:58:44.041 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:44.041 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:44.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:44.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:44.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:58:44.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:58:44.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:58:44.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:58:44.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:58:44.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:58:44.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:58:44.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:58:44.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:58:44.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:58:44.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:58:44.044 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:58:44.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:58:44.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:58:44.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:58:44.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:58:44.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:58:44.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:58:44.059 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:44.058+0000 7f8fd45f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:44.061 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:44.058+0000 7f8fd45f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:44.062 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:44.058+0000 7f8fd45f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:58:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:58:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:58:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:58:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:58:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:58:44.205 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:58:44.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:44.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:58:44.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:44.496 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:44.494+0000 7f8fd45f1a40 -1 Falling back to public interface 2026-02-19T12:58:45.077 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:45.074+0000 7f8fd45f1a40 -1 osd.1 37 log_to_monitors true 2026-02-19T12:58:45.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:45.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:45.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:58:45.366 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:58:45.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:45.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:58:45.527 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 41 up_thru 41 down_at 38 last_clean_interval [35,37) [v2:127.0.0.1:6810/3303317061,v1:127.0.0.1:6811/3303317061] [v2:127.0.0.1:6812/3303317061,v1:127.0.0.1:6813/3303317061] exists,up a3608c17-20f6-4aba-9382-efddb645ce47 2026-02-19T12:58:45.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:58:45.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:58:45.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:58:45.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:58:45.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:58:45.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:58:45.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:58:45.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:58:45.529 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:58:45.529 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:58:45.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:58:45.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:58:45.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:58:45.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:58:45.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:58:45.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:58:45.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:58:45.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:58:45.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:58:45.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:58:45.804 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T12:58:45.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:58:45.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:45.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:58:45.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280521 2026-02-19T12:58:45.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280521 2026-02-19T12:58:45.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-94489280521' 2026-02-19T12:58:45.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:58:45.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:58:45.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659138 2026-02-19T12:58:45.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659138 2026-02-19T12:58:45.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-94489280521 1-176093659138' 2026-02-19T12:58:45.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:45.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-94489280521 2026-02-19T12:58:45.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:45.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:58:45.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-94489280521 2026-02-19T12:58:45.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:45.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=94489280521 2026-02-19T12:58:45.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 94489280521' 2026-02-19T12:58:45.959 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 94489280521 2026-02-19T12:58:45.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:58:46.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280520 -lt 94489280521 2026-02-19T12:58:46.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:58:47.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:58:47.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:58:47.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280520 -lt 94489280521 2026-02-19T12:58:47.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:58:48.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:58:48.291 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:58:48.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 94489280522 -lt 94489280521 2026-02-19T12:58:48.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:58:48.452 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-176093659138 2026-02-19T12:58:48.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:58:48.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:58:48.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-176093659138 2026-02-19T12:58:48.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:58:48.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=176093659138 2026-02-19T12:58:48.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 176093659138' 2026-02-19T12:58:48.457 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 176093659138 2026-02-19T12:58:48.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:58:48.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 176093659138 -lt 176093659138 2026-02-19T12:58:48.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:58:48.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:58:48.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:58:48.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T12:58:48.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:58:48.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:58:48.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:58:48.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:58:48.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:58:48.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:58:48.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:58:48.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T12:58:48.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:58:48.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:58:48.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:58:49.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T12:58:49.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:58:49.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:58:49.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: corrupt_and_repair_one: rados --pool rbd get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T12:58:49.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:304: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T12:58:49.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T12:58:49.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:58:49.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:58:49.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:58:49.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:58:49.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:58:49.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:58:49.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:58:49.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:58:49.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:58:49.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:58:49.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:58:49.298 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:58:49.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:58:49.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:58:49.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:58:49.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:58:49.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:58:49.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:58:49.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:58:49.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:58:49.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:58:49.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:58:49.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:58:49.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:49.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:49.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:58:49.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:58:49.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:58:49.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T12:58:49.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T12:58:49.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T12:58:49.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T12:58:49.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T12:58:49.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T12:58:49.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T12:58:49.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:58:49.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:58:49.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:58:49.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:58:49.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:58:49.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:58:49.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T12:58:49.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T12:58:49.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T12:58:49.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T12:58:49.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T12:58:49.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T12:58:49.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:58:49.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T12:58:49.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T12:58:49.328 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T12:58:49.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T12:58:49.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T12:58:49.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T12:58:49.331 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T12:58:49.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:49.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:49.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T12:58:49.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T12:58:49.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T12:58:49.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T12:58:49.335 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T12:58:49.335 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:49.335 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:49.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T12:58:49.337 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T12:58:49.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T12:58:49.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T12:58:49.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T12:58:49.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_scrub_erasure_appends td/osd-scrub-repair 2026-02-19T12:58:49.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5912: TEST_corrupt_scrub_erasure_appends: corrupt_scrub_erasure td/osd-scrub-repair false 2026-02-19T12:58:49.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3736: corrupt_scrub_erasure: local dir=td/osd-scrub-repair 2026-02-19T12:58:49.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3737: corrupt_scrub_erasure: local allow_overwrites=false 2026-02-19T12:58:49.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3738: corrupt_scrub_erasure: local poolname=ecpool 2026-02-19T12:58:49.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3739: corrupt_scrub_erasure: local total_objs=7 2026-02-19T12:58:49.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3741: corrupt_scrub_erasure: run_mon td/osd-scrub-repair a 2026-02-19T12:58:49.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T12:58:49.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T12:58:49.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T12:58:49.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T12:58:49.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T12:58:49.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T12:58:49.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T12:58:49.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:58:49.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:58:49.359 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:58:49.359 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:49.359 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:49.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:49.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T12:58:49.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T12:58:49.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T12:58:49.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:58:49.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:58:49.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T12:58:49.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T12:58:49.393 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:58:49.393 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:58:49.393 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:58:49.393 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:58:49.393 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:49.393 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:49.393 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:58:49.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:58:49.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T12:58:49.447 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T12:58:49.448 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T12:58:49.448 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T12:58:49.448 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T12:58:49.448 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T12:58:49.449 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T12:58:49.449 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T12:58:49.449 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T12:58:49.450 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T12:58:49.450 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:49.450 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:49.450 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T12:58:49.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T12:58:49.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T12:58:49.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3742: corrupt_scrub_erasure: run_mgr td/osd-scrub-repair x 2026-02-19T12:58:49.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T12:58:49.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T12:58:49.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T12:58:49.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T12:58:49.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T12:58:49.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T12:58:49.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T12:58:49.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:58:49.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:58:49.596 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:58:49.596 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:49.596 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:49.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:49.597 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:58:49.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T12:58:49.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3743: corrupt_scrub_erasure: seq 0 2 2026-02-19T12:58:49.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3743: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-19T12:58:49.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3744: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 0 2026-02-19T12:58:49.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:58:49.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:58:49.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T12:58:49.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:58:49.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:58:49.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:58:49.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:58:49.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:58:49.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:58:49.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:58:49.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:58:49.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:58:49.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:58:49.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:58:49.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:58:49.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:58:49.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:58:49.621 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:58:49.622 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:49.622 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:49.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:49.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:49.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:58:49.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:58:49.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:58:49.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:58:49.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:58:49.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:58:49.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:58:49.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:58:49.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:58:49.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:58:49.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:58:49.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:58:49.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:58:49.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a97b81ca-b503-46b6-9623-80a650d2efd1 2026-02-19T12:58:49.624 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 a97b81ca-b503-46b6-9623-80a650d2efd1 2026-02-19T12:58:49.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 a97b81ca-b503-46b6-9623-80a650d2efd1' 2026-02-19T12:58:49.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:58:49.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAJCZdp37LJJRAA0Klxjd73nEHmo49Ut0UTTA== 2026-02-19T12:58:49.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAJCZdp37LJJRAA0Klxjd73nEHmo49Ut0UTTA=="}' 2026-02-19T12:58:49.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a97b81ca-b503-46b6-9623-80a650d2efd1 -i td/osd-scrub-repair/0/new.json 2026-02-19T12:58:49.725 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:58:49.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T12:58:49.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAJCZdp37LJJRAA0Klxjd73nEHmo49Ut0UTTA== --osd-uuid a97b81ca-b503-46b6-9623-80a650d2efd1 2026-02-19T12:58:49.750 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:49.746+0000 7f3cbf96fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:49.751 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:49.750+0000 7f3cbf96fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:49.752 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:49.750+0000 7f3cbf96fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:49.753 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:49.750+0000 7f3cbf96fa40 -1 bdev(0x561b07f70c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T12:58:49.753 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:49.750+0000 7f3cbf96fa40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T12:58:50.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T12:58:50.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:58:50.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T12:58:50.471 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T12:58:50.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:58:50.678 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T12:58:50.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T12:58:50.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:58:50.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:58:50.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:58:50.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:58:50.694 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:50.690+0000 7fd62bf59a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:50.696 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:50.694+0000 7fd62bf59a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:50.697 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:50.694+0000 7fd62bf59a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:50.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T12:58:50.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:58:50.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:58:50.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:58:50.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:58:50.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:50.839 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:58:50.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:58:50.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:50.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:58:50.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:51.024 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:51.022+0000 7fd62bf59a40 -1 Falling back to public interface 2026-02-19T12:58:51.604 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:51.602+0000 7fd62bf59a40 -1 osd.0 0 log_to_monitors true 2026-02-19T12:58:51.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:51.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:51.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:58:51.996 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:58:51.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:51.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:58:52.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:53.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:53.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:53.156 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:58:53.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:58:53.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:53.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:58:53.316 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/809566715,v1:127.0.0.1:6803/809566715] [v2:127.0.0.1:6804/809566715,v1:127.0.0.1:6805/809566715] exists,up a97b81ca-b503-46b6-9623-80a650d2efd1 2026-02-19T12:58:53.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:58:53.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:58:53.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:58:53.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3743: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-19T12:58:53.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3744: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 1 2026-02-19T12:58:53.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:58:53.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:58:53.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T12:58:53.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:58:53.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:58:53.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:58:53.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:58:53.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:58:53.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:58:53.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:58:53.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:58:53.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:58:53.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:58:53.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:58:53.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:58:53.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:58:53.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:58:53.321 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:58:53.321 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:53.321 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:53.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:53.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:53.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:58:53.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:58:53.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:58:53.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:58:53.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:58:53.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:58:53.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:58:53.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:58:53.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:58:53.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:58:53.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:58:53.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:58:53.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:58:53.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=063d83eb-ff81-4f56-b415-8fbd1f91414f 2026-02-19T12:58:53.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 063d83eb-ff81-4f56-b415-8fbd1f91414f' 2026-02-19T12:58:53.323 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 063d83eb-ff81-4f56-b415-8fbd1f91414f 2026-02-19T12:58:53.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:58:53.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQANCZdpoaj0ExAA5NfMZ6QpCSDnOrcgbxx/GQ== 2026-02-19T12:58:53.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQANCZdpoaj0ExAA5NfMZ6QpCSDnOrcgbxx/GQ=="}' 2026-02-19T12:58:53.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 063d83eb-ff81-4f56-b415-8fbd1f91414f -i td/osd-scrub-repair/1/new.json 2026-02-19T12:58:53.496 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:58:53.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T12:58:53.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQANCZdpoaj0ExAA5NfMZ6QpCSDnOrcgbxx/GQ== --osd-uuid 063d83eb-ff81-4f56-b415-8fbd1f91414f 2026-02-19T12:58:53.523 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:53.522+0000 7f84e11cfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:53.525 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:53.522+0000 7f84e11cfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:53.526 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:53.522+0000 7f84e11cfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:53.527 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:53.522+0000 7f84e11cfa40 -1 bdev(0x55e11e192c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T12:58:53.527 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:53.522+0000 7f84e11cfa40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T12:58:54.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T12:58:54.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:58:54.171 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T12:58:54.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T12:58:54.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:58:54.376 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T12:58:54.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T12:58:54.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:58:54.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:58:54.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:58:54.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:58:54.400 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:54.398+0000 7f10783aca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:54.402 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:54.398+0000 7f10783aca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:54.403 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:54.402+0000 7f10783aca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:54.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T12:58:54.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:58:54.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:58:54.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:58:54.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:58:54.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:54.536 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:58:54.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:58:54.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:54.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:58:54.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:54.804 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:54.802+0000 7f10783aca40 -1 Falling back to public interface 2026-02-19T12:58:55.427 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:55.422+0000 7f10783aca40 -1 osd.1 0 log_to_monitors true 2026-02-19T12:58:55.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:55.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:55.697 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:58:55.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:58:55.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:55.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:58:55.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:56.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:56.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:56.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:58:56.855 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:58:56.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:56.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:58:57.011 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3136247782,v1:127.0.0.1:6811/3136247782] [v2:127.0.0.1:6812/3136247782,v1:127.0.0.1:6813/3136247782] exists,up 063d83eb-ff81-4f56-b415-8fbd1f91414f 2026-02-19T12:58:57.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:58:57.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:58:57.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:58:57.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3743: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-19T12:58:57.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3744: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 2 2026-02-19T12:58:57.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T12:58:57.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T12:58:57.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T12:58:57.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T12:58:57.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:58:57.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:58:57.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:58:57.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:58:57.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:58:57.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:58:57.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:58:57.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T12:58:57.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T12:58:57.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:58:57.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T12:58:57.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:58:57.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:58:57.016 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:58:57.017 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:58:57.017 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:58:57.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:57.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:58:57.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:58:57.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T12:58:57.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T12:58:57.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:58:57.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:58:57.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:58:57.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:58:57.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:58:57.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:58:57.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T12:58:57.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T12:58:57.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:58:57.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T12:58:57.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=61dd9924-18ad-468d-a81d-ab2d915ddbd0 2026-02-19T12:58:57.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 61dd9924-18ad-468d-a81d-ab2d915ddbd0' 2026-02-19T12:58:57.019 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 61dd9924-18ad-468d-a81d-ab2d915ddbd0 2026-02-19T12:58:57.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T12:58:57.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQARCZdp/WP2ARAApwRMmjzkgPSAMgY+onja/g== 2026-02-19T12:58:57.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQARCZdp/WP2ARAApwRMmjzkgPSAMgY+onja/g=="}' 2026-02-19T12:58:57.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 61dd9924-18ad-468d-a81d-ab2d915ddbd0 -i td/osd-scrub-repair/2/new.json 2026-02-19T12:58:57.185 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:58:57.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T12:58:57.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQARCZdp/WP2ARAApwRMmjzkgPSAMgY+onja/g== --osd-uuid 61dd9924-18ad-468d-a81d-ab2d915ddbd0 2026-02-19T12:58:57.220 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:57.218+0000 7ff6152fba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:57.222 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:57.218+0000 7ff6152fba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:57.223 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:57.222+0000 7ff6152fba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:57.223 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:57.222+0000 7ff6152fba40 -1 bdev(0x561e98b16c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T12:58:57.223 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:57.222+0000 7ff6152fba40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T12:58:57.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T12:58:57.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T12:58:57.855 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T12:58:57.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T12:58:57.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T12:58:58.055 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T12:58:58.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T12:58:58.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:58:58.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T12:58:58.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T12:58:58.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T12:58:58.079 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:58.078+0000 7f2afbc7ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:58.081 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:58.078+0000 7f2afbc7ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:58.082 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:58.082+0000 7f2afbc7ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:58:58.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T12:58:58.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:58:58.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:58:58.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:58:58.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:58:58.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:58.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:58:58.221 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T12:58:58.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:58.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:58:58.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:58:58.496 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:58.494+0000 7f2afbc7ca40 -1 Falling back to public interface 2026-02-19T12:58:59.069 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:58:59.066+0000 7f2afbc7ca40 -1 osd.2 0 log_to_monitors true 2026-02-19T12:58:59.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:58:59.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:58:59.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:58:59.382 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T12:58:59.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:58:59.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:58:59.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:59:00.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:59:00.540 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T12:59:00.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:00.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T12:59:00.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:00.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:59:00.696 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1014273678,v1:127.0.0.1:6819/1014273678] [v2:127.0.0.1:6820/1014273678,v1:127.0.0.1:6821/1014273678] exists,up 61dd9924-18ad-468d-a81d-ab2d915ddbd0 2026-02-19T12:59:00.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:59:00.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:59:00.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:59:00.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3746: corrupt_scrub_erasure: create_rbd_pool 2026-02-19T12:59:00.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T12:59:00.844 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T12:59:00.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T12:59:00.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T12:59:01.059 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T12:59:01.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:59:02.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T12:59:02.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3747: corrupt_scrub_erasure: create_pool foo 1 2026-02-19T12:59:02.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 1 2026-02-19T12:59:02.547 INFO:tasks.workunit.client.0.trial144.stderr:pool 'foo' created 2026-02-19T12:59:02.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:59:03.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3749: corrupt_scrub_erasure: create_ec_pool ecpool false k=2 m=1 stripe_unit=2K --force 2026-02-19T12:59:03.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: create_ec_pool: local pool_name=ecpool 2026-02-19T12:59:03.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: create_ec_pool: shift 2026-02-19T12:59:03.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: create_ec_pool: local allow_overwrites=false 2026-02-19T12:59:03.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2525: create_ec_pool: shift 2026-02-19T12:59:03.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 stripe_unit=2K --force 2026-02-19T12:59:03.763 INFO:tasks.workunit.client.0.trial144.stderr:load: isa 2026-02-19T12:59:03.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-19T12:59:03.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-19T12:59:04.068 INFO:tasks.workunit.client.0.trial144.stderr:pool 'ecpool' created 2026-02-19T12:59:04.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T12:59:05.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: create_ec_pool: '[' false = true ']' 2026-02-19T12:59:05.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2535: create_ec_pool: wait_for_clean 2026-02-19T12:59:05.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:59:05.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:59:05.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:59:05.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:59:05.082 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:59:05.082 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:59:05.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:59:05.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:59:05.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:59:05.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:59:05.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:59:05.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:59:05.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:59:05.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:59:05.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:59:05.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:59:05.360 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:05.360 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:59:05.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:59:05.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:05.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:59:05.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-19T12:59:05.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-19T12:59:05.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-19T12:59:05.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:05.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:59:05.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-19T12:59:05.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-19T12:59:05.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-38654705667' 2026-02-19T12:59:05.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:05.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:59:05.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542147 2026-02-19T12:59:05.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542147 2026-02-19T12:59:05.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-38654705667 2-60129542147' 2026-02-19T12:59:05.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:05.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-19T12:59:05.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:05.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:59:05.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-19T12:59:05.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:05.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-19T12:59:05.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-19T12:59:05.583 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836484 2026-02-19T12:59:05.583 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:05.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-19T12:59:05.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:06.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:59:06.754 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:06.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-19T12:59:06.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:06.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-19T12:59:06.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:06.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:59:06.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705667 2026-02-19T12:59:06.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:06.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705667 2026-02-19T12:59:06.919 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705667 2026-02-19T12:59:06.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-19T12:59:06.919 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:59:07.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705667 -lt 38654705667 2026-02-19T12:59:07.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:07.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542147 2026-02-19T12:59:07.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:07.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:59:07.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542147 2026-02-19T12:59:07.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:07.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542147 2026-02-19T12:59:07.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-19T12:59:07.080 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542147 2026-02-19T12:59:07.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:59:07.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-19T12:59:07.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:59:07.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:07.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:07.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T12:59:07.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:59:07.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:59:07.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:59:07.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:59:07.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:59:07.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:59:07.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:59:07.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T12:59:07.601 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:59:07.601 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:07.601 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:07.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T12:59:07.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:59:07.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:59:07.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2536: create_ec_pool: return 0 2026-02-19T12:59:07.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3750: corrupt_scrub_erasure: wait_for_clean 2026-02-19T12:59:07.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:59:07.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:59:07.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:59:07.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:59:07.796 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:59:07.796 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:59:07.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:59:07.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:59:07.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:59:07.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:59:07.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:59:07.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:59:07.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:59:07.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:59:07.903 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:59:08.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:59:08.059 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:08.060 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:59:08.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:59:08.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:08.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:59:08.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836486 2026-02-19T12:59:08.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836486 2026-02-19T12:59:08.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836486' 2026-02-19T12:59:08.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:08.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:59:08.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705669 2026-02-19T12:59:08.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705669 2026-02-19T12:59:08.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836486 1-38654705669' 2026-02-19T12:59:08.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:08.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:59:08.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542148 2026-02-19T12:59:08.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542148 2026-02-19T12:59:08.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836486 1-38654705669 2-60129542148' 2026-02-19T12:59:08.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:08.279 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836486 2026-02-19T12:59:08.279 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:08.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:59:08.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836486 2026-02-19T12:59:08.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:08.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836486 2026-02-19T12:59:08.283 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836486 2026-02-19T12:59:08.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-19T12:59:08.284 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:08.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-19T12:59:08.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:09.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:59:09.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:09.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-19T12:59:09.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:10.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:59:10.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:10.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-19T12:59:10.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:10.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705669 2026-02-19T12:59:10.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:10.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:59:10.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705669 2026-02-19T12:59:10.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:10.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705669 2026-02-19T12:59:10.786 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705669 2026-02-19T12:59:10.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-19T12:59:10.786 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:59:10.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-19T12:59:10.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:10.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542148 2026-02-19T12:59:10.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:10.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:59:10.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542148 2026-02-19T12:59:10.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:10.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542148 2026-02-19T12:59:10.958 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542148 2026-02-19T12:59:10.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-19T12:59:10.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:59:11.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-19T12:59:11.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:59:11.131 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:11.131 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:11.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T12:59:11.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:59:11.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:59:11.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:59:11.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:59:11.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:59:11.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:59:11.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:59:11.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T12:59:11.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:59:11.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:11.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:11.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T12:59:11.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:59:11.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:59:11.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: seq 1 7 2026-02-19T12:59:11.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-19T12:59:11.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3753: corrupt_scrub_erasure: objname=EOBJ1 2026-02-19T12:59:11.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3754: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ1 2026-02-19T12:59:11.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T12:59:11.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T12:59:11.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ1 2026-02-19T12:59:11.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T12:59:11.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T12:59:11.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T12:59:11.872 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T12:59:11.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T12:59:12.074 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T12:59:12.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T12:59:12.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T12:59:12.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:59:12.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: expr 1 % 2 2026-02-19T12:59:12.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: local osd=1 2026-02-19T12:59:12.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3758: corrupt_scrub_erasure: case $i in 2026-02-19T12:59:12.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3761: corrupt_scrub_erasure: local payload=UVWXYZZZ 2026-02-19T12:59:12.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3762: corrupt_scrub_erasure: echo UVWXYZZZ 2026-02-19T12:59:12.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3763: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T12:59:12.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:59:12.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:59:12.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:59:12.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:59:12.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T12:59:12.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:59:12.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:59:12.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:59:12.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:59:12.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:59:12.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:59:12.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:59:12.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:59:12.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:59:12.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:59:12.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:59:12.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T12:59:12.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:59:12.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:59:12.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:59:12.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:59:12.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:59:12.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T12:59:12.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:59:12.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:59:12.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:59:12.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:59:12.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:59:12.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:59:12.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:59:12.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:59:12.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:59:12.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:59:12.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:59:12.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:59:12.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:59:12.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:59:12.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:59:12.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:59:12.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:59:12.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:59:12.728 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:59:12.728 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:59:12.728 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:59:12.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:12.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:12.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:59:12.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:59:12.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:59:12.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:59:12.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:59:12.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:59:12.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:59:12.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:59:12.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:59:12.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:59:12.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:59:12.731 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:59:12.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:59:12.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:59:12.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:59:12.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:59:12.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:59:12.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:59:12.744 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:12.742+0000 7f8568188a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:12.745 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:12.742+0000 7f8568188a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:12.746 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:12.746+0000 7f8568188a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:12.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:59:12.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:59:12.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:59:12.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:59:12.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:59:12.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:12.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:59:12.888 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:59:12.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:12.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:59:13.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:59:13.108 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:13.106+0000 7f8568188a40 -1 Falling back to public interface 2026-02-19T12:59:13.737 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:13.734+0000 7f8568188a40 -1 osd.1 32 log_to_monitors true 2026-02-19T12:59:14.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:59:14.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:14.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:59:14.050 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:14.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:14.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:59:14.211 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 36 up_thru 36 down_at 33 last_clean_interval [9,32) [v2:127.0.0.1:6810/1367225405,v1:127.0.0.1:6811/1367225405] [v2:127.0.0.1:6812/1367225405,v1:127.0.0.1:6813/1367225405] exists,up 063d83eb-ff81-4f56-b415-8fbd1f91414f 2026-02-19T12:59:14.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:59:14.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:59:14.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:59:14.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:59:14.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:59:14.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:59:14.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:59:14.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:59:14.213 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:59:14.213 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:59:14.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:59:14.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:59:14.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:59:14.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:59:14.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:59:14.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:59:14.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:59:14.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:59:14.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:59:14.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:59:14.481 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:14.481 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:59:14.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:59:14.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:14.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:59:14.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836488 2026-02-19T12:59:14.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836488 2026-02-19T12:59:14.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488' 2026-02-19T12:59:14.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:14.554 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:59:14.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822658 2026-02-19T12:59:14.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822658 2026-02-19T12:59:14.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-154618822658' 2026-02-19T12:59:14.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:14.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:59:14.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542150 2026-02-19T12:59:14.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542150 2026-02-19T12:59:14.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-154618822658 2-60129542150' 2026-02-19T12:59:14.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:14.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836488 2026-02-19T12:59:14.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:14.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:59:14.700 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836488 2026-02-19T12:59:14.700 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:14.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836488 2026-02-19T12:59:14.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-19T12:59:14.702 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836488 2026-02-19T12:59:14.702 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:14.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836488 2026-02-19T12:59:14.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:15.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:59:15.877 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:16.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836488 2026-02-19T12:59:16.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:17.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:59:17.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:17.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-19T12:59:17.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:17.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-154618822658 2026-02-19T12:59:17.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:17.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:59:17.200 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-154618822658 2026-02-19T12:59:17.200 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:17.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=154618822658 2026-02-19T12:59:17.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 154618822658' 2026-02-19T12:59:17.202 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 154618822658 2026-02-19T12:59:17.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:59:17.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 154618822658 -lt 154618822658 2026-02-19T12:59:17.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:17.369 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542150 2026-02-19T12:59:17.370 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:17.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:59:17.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542150 2026-02-19T12:59:17.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:17.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542150 2026-02-19T12:59:17.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-19T12:59:17.373 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542150 2026-02-19T12:59:17.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:59:17.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542151 -lt 60129542150 2026-02-19T12:59:17.537 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:59:17.538 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:17.538 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:17.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T12:59:17.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:59:17.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:59:17.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:59:17.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:59:17.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:59:17.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:59:17.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:59:17.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T12:59:17.883 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:59:17.883 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:17.883 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:18.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T12:59:18.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:59:18.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:59:18.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-19T12:59:18.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3753: corrupt_scrub_erasure: objname=EOBJ2 2026-02-19T12:59:18.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3754: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ2 2026-02-19T12:59:18.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T12:59:18.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T12:59:18.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ2 2026-02-19T12:59:18.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T12:59:18.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T12:59:18.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T12:59:18.242 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T12:59:18.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T12:59:18.444 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T12:59:18.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T12:59:18.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T12:59:18.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ2 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:59:18.477 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: expr 2 % 2 2026-02-19T12:59:18.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: local osd=0 2026-02-19T12:59:18.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3758: corrupt_scrub_erasure: case $i in 2026-02-19T12:59:18.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3768: corrupt_scrub_erasure: dd if=/dev/urandom of=td/osd-scrub-repair/CORRUPT bs=2048 count=1 2026-02-19T12:59:18.480 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T12:59:18.481 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T12:59:18.481 INFO:tasks.workunit.client.0.trial144.stderr:2048 bytes (2.0 kB, 2.0 KiB) copied, 6.9549e-05 s, 29.4 MB/s 2026-02-19T12:59:18.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3769: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T12:59:18.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:59:18.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:59:18.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:59:18.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:59:18.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T12:59:18.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:59:18.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:59:18.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:59:18.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:59:18.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:59:18.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:59:18.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:59:18.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:59:18.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:59:18.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:59:18.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:59:18.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T12:59:18.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:59:18.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:59:18.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:59:18.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:59:18.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:59:18.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T12:59:19.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T12:59:19.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:59:19.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:59:19.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:59:19.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:59:19.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:59:19.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:59:19.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:59:19.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:59:19.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:59:19.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:59:19.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:59:19.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:59:19.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:59:19.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:59:19.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:59:19.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:59:19.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:59:19.079 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:59:19.079 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:59:19.079 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:59:19.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:19.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:19.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:59:19.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:59:19.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:59:19.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:59:19.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:59:19.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:59:19.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:59:19.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:59:19.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:59:19.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:59:19.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:59:19.082 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:59:19.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:59:19.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:59:19.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:59:19.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:59:19.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:59:19.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:59:19.098 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:19.093+0000 7fa63e163a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:19.100 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:19.097+0000 7fa63e163a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:19.101 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:19.097+0000 7fa63e163a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:19.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:59:19.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:59:19.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:59:19.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:59:19.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:59:19.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:19.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:59:19.240 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:59:19.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:19.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:59:19.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:59:19.512 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:19.509+0000 7fa63e163a40 -1 Falling back to public interface 2026-02-19T12:59:20.070 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:20.065+0000 7fa63e163a40 -1 osd.0 39 log_to_monitors true 2026-02-19T12:59:20.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:59:20.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:20.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:59:20.403 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:20.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:20.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:59:20.565 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 43 up_thru 43 down_at 40 last_clean_interval [5,39) [v2:127.0.0.1:6802/122205386,v1:127.0.0.1:6803/122205386] [v2:127.0.0.1:6804/122205386,v1:127.0.0.1:6805/122205386] exists,up a97b81ca-b503-46b6-9623-80a650d2efd1 2026-02-19T12:59:20.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:59:20.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:59:20.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:59:20.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:59:20.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:59:20.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:59:20.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:59:20.566 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:59:20.567 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:59:20.567 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:59:20.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:59:20.568 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:59:20.568 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:59:20.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:59:20.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:59:20.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:59:20.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:59:20.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:59:20.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:59:20.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:59:20.839 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:20.839 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:59:20.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:59:20.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:20.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:59:20.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593730 2026-02-19T12:59:20.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593730 2026-02-19T12:59:20.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-184683593730' 2026-02-19T12:59:20.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:20.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:59:20.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822660 2026-02-19T12:59:20.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822660 2026-02-19T12:59:20.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-184683593730 1-154618822660' 2026-02-19T12:59:20.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:20.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:59:21.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542153 2026-02-19T12:59:21.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542153 2026-02-19T12:59:21.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-184683593730 1-154618822660 2-60129542153' 2026-02-19T12:59:21.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:21.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-184683593730 2026-02-19T12:59:21.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:21.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:59:21.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-184683593730 2026-02-19T12:59:21.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:21.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593730 2026-02-19T12:59:21.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 184683593730' 2026-02-19T12:59:21.070 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 184683593730 2026-02-19T12:59:21.070 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:21.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836489 -lt 184683593730 2026-02-19T12:59:21.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:22.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:59:22.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:22.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593730 -lt 184683593730 2026-02-19T12:59:22.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:22.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-154618822660 2026-02-19T12:59:22.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:22.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:59:22.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-154618822660 2026-02-19T12:59:22.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:22.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=154618822660 2026-02-19T12:59:22.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 154618822660' 2026-02-19T12:59:22.409 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 154618822660 2026-02-19T12:59:22.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:59:22.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 154618822660 -lt 154618822660 2026-02-19T12:59:22.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:22.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542153 2026-02-19T12:59:22.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:22.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:59:22.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542153 2026-02-19T12:59:22.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:22.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542153 2026-02-19T12:59:22.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-19T12:59:22.583 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542153 2026-02-19T12:59:22.584 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:59:22.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-19T12:59:22.744 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:59:22.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:22.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:22.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T12:59:22.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:59:22.934 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:59:22.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:59:22.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:59:22.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:59:22.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:59:22.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:59:23.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T12:59:23.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:59:23.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:23.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:23.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T12:59:23.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:59:23.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:59:23.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-19T12:59:23.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3753: corrupt_scrub_erasure: objname=EOBJ3 2026-02-19T12:59:23.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3754: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ3 2026-02-19T12:59:23.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T12:59:23.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T12:59:23.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ3 2026-02-19T12:59:23.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T12:59:23.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T12:59:23.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T12:59:23.484 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T12:59:23.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T12:59:23.687 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T12:59:23.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T12:59:23.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T12:59:23.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ3 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:59:23.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: expr 3 % 2 2026-02-19T12:59:23.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: local osd=1 2026-02-19T12:59:23.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3758: corrupt_scrub_erasure: case $i in 2026-02-19T12:59:23.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3774: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ3 remove 2026-02-19T12:59:23.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:59:23.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:59:23.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:59:23.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:59:23.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ3 remove 2026-02-19T12:59:23.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:59:23.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:59:23.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:59:23.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:59:23.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:59:23.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:59:23.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:59:23.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:59:23.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:59:23.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:59:24.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:59:24.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ3 remove 2026-02-19T12:59:24.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:59:24.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:59:24.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:59:24.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:59:24.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:59:24.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ3 remove 2026-02-19T12:59:24.234 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#3:b197b25d:::EOBJ3:head# 2026-02-19T12:59:24.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:59:24.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:59:24.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:59:24.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:59:24.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:59:24.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:59:24.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:59:24.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:59:24.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:59:24.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:59:24.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:59:24.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:59:24.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:59:24.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:59:24.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:59:24.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:59:24.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:59:24.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:59:24.327 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:59:24.327 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:59:24.327 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:59:24.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:24.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:24.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:59:24.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:59:24.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:59:24.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:59:24.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:59:24.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:59:24.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:59:24.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:59:24.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:59:24.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:59:24.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:59:24.330 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:59:24.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:59:24.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:59:24.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:59:24.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:59:24.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:59:24.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:59:24.346 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:24.341+0000 7fe684923a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:24.347 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:24.345+0000 7fe684923a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:24.348 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:24.345+0000 7fe684923a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:24.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:59:24.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:59:24.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:59:24.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:59:24.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:59:24.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:24.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:59:24.490 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:59:24.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:24.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:59:24.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:59:24.772 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:24.769+0000 7fe684923a40 -1 Falling back to public interface 2026-02-19T12:59:25.373 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:25.369+0000 7fe684923a40 -1 osd.1 47 log_to_monitors true 2026-02-19T12:59:25.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:59:25.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:25.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:59:25.648 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:25.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:25.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:59:25.804 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 51 up_thru 51 down_at 48 last_clean_interval [36,47) [v2:127.0.0.1:6810/943160737,v1:127.0.0.1:6811/943160737] [v2:127.0.0.1:6812/943160737,v1:127.0.0.1:6813/943160737] exists,up 063d83eb-ff81-4f56-b415-8fbd1f91414f 2026-02-19T12:59:25.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:59:25.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:59:25.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:59:25.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:59:25.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:59:25.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:59:25.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:59:25.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:59:25.806 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:59:25.806 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:59:25.806 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:59:25.806 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:59:25.806 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:59:25.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:59:25.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:59:25.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:59:25.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:59:25.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:59:25.921 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:59:26.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:59:26.080 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:26.080 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:59:26.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:59:26.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:26.080 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:59:26.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593732 2026-02-19T12:59:26.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593732 2026-02-19T12:59:26.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-184683593732' 2026-02-19T12:59:26.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:26.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:59:26.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332098 2026-02-19T12:59:26.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332098 2026-02-19T12:59:26.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-184683593732 1-219043332098' 2026-02-19T12:59:26.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:26.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:59:26.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542155 2026-02-19T12:59:26.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542155 2026-02-19T12:59:26.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-184683593732 1-219043332098 2-60129542155' 2026-02-19T12:59:26.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:26.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-184683593732 2026-02-19T12:59:26.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:26.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:59:26.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-184683593732 2026-02-19T12:59:26.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:26.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593732 2026-02-19T12:59:26.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 184683593732' 2026-02-19T12:59:26.302 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 184683593732 2026-02-19T12:59:26.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:26.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593731 -lt 184683593732 2026-02-19T12:59:26.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:27.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:59:27.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:27.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593731 -lt 184683593732 2026-02-19T12:59:27.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:28.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:59:28.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:28.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593732 -lt 184683593732 2026-02-19T12:59:28.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:28.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-219043332098 2026-02-19T12:59:28.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:28.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:59:28.794 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-219043332098 2026-02-19T12:59:28.794 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:28.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=219043332098 2026-02-19T12:59:28.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 219043332098' 2026-02-19T12:59:28.796 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 219043332098 2026-02-19T12:59:28.796 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:59:28.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 219043332098 -lt 219043332098 2026-02-19T12:59:28.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:28.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542155 2026-02-19T12:59:28.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:28.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:59:28.967 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542155 2026-02-19T12:59:28.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:28.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542155 2026-02-19T12:59:28.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542155' 2026-02-19T12:59:28.969 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542155 2026-02-19T12:59:28.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:59:29.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542155 -lt 60129542155 2026-02-19T12:59:29.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:59:29.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:29.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:29.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T12:59:29.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:59:29.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:59:29.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:59:29.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:59:29.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:59:29.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:59:29.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:59:29.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T12:59:29.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:59:29.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:29.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:29.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T12:59:29.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:59:29.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:59:29.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-19T12:59:29.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3753: corrupt_scrub_erasure: objname=EOBJ4 2026-02-19T12:59:29.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3754: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ4 2026-02-19T12:59:29.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T12:59:29.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T12:59:29.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ4 2026-02-19T12:59:29.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T12:59:29.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T12:59:29.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T12:59:29.881 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T12:59:29.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T12:59:30.086 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T12:59:30.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T12:59:30.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T12:59:30.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ4 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:59:30.123 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: expr 4 % 2 2026-02-19T12:59:30.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: local osd=0 2026-02-19T12:59:30.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3758: corrupt_scrub_erasure: case $i in 2026-02-19T12:59:30.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3778: corrupt_scrub_erasure: rados --pool ecpool setxattr EOBJ4 key1-EOBJ4 val1-EOBJ4 2026-02-19T12:59:30.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3779: corrupt_scrub_erasure: rados --pool ecpool setxattr EOBJ4 key2-EOBJ4 val2-EOBJ4 2026-02-19T12:59:30.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3782: corrupt_scrub_erasure: echo -n bad-val 2026-02-19T12:59:30.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3783: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-19T12:59:30.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:59:30.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:59:30.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:59:30.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:59:30.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-19T12:59:30.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:59:30.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:59:30.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:59:30.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:59:30.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:59:30.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:59:30.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:59:30.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:59:30.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:59:30.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:59:30.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:59:30.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-19T12:59:30.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:59:30.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:59:30.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:59:30.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:59:30.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:59:30.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-19T12:59:30.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T12:59:30.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:59:30.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:59:30.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:59:30.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:59:30.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:59:30.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:59:30.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:59:30.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:59:30.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:59:30.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:59:30.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:59:30.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:59:30.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:59:30.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:59:30.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:59:30.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:59:30.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:59:30.767 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:59:30.767 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:59:30.767 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:59:30.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:30.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:30.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:59:30.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:59:30.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:59:30.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:59:30.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:59:30.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:59:30.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:59:30.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:59:30.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:59:30.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:59:30.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:59:30.769 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:59:30.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:59:30.770 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:59:30.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:59:30.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:59:30.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:59:30.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:59:30.784 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:30.781+0000 7f42a72d3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:30.785 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:30.781+0000 7f42a72d3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:30.786 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:30.781+0000 7f42a72d3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:30.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:59:30.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:59:30.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:59:30.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:59:30.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:59:30.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:30.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:59:30.930 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:59:30.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:30.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:59:31.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:59:31.200 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:31.197+0000 7f42a72d3a40 -1 Falling back to public interface 2026-02-19T12:59:31.757 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:31.753+0000 7f42a72d3a40 -1 osd.0 54 log_to_monitors true 2026-02-19T12:59:32.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:59:32.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:32.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:59:32.090 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:32.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:32.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:59:32.248 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 58 up_thru 58 down_at 55 last_clean_interval [43,54) [v2:127.0.0.1:6802/4243964765,v1:127.0.0.1:6803/4243964765] [v2:127.0.0.1:6804/4243964765,v1:127.0.0.1:6805/4243964765] exists,up a97b81ca-b503-46b6-9623-80a650d2efd1 2026-02-19T12:59:32.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:59:32.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:59:32.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:59:32.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:59:32.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:59:32.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:59:32.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:59:32.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:59:32.251 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:59:32.251 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:59:32.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:59:32.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:59:32.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:59:32.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:59:32.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:59:32.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:59:32.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:59:32.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:59:32.364 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:59:32.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:59:32.519 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:32.519 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:59:32.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:59:32.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:32.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:59:32.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=249108103170 2026-02-19T12:59:32.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 249108103170 2026-02-19T12:59:32.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103170' 2026-02-19T12:59:32.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:32.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:59:32.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332100 2026-02-19T12:59:32.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332100 2026-02-19T12:59:32.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103170 1-219043332100' 2026-02-19T12:59:32.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:32.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:59:32.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542157 2026-02-19T12:59:32.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542157 2026-02-19T12:59:32.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103170 1-219043332100 2-60129542157' 2026-02-19T12:59:32.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:32.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-249108103170 2026-02-19T12:59:32.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:32.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:59:32.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-249108103170 2026-02-19T12:59:32.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:32.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=249108103170 2026-02-19T12:59:32.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 249108103170' 2026-02-19T12:59:32.734 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 249108103170 2026-02-19T12:59:32.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:32.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 249108103170 2026-02-19T12:59:32.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:33.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:59:33.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:34.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 249108103170 2026-02-19T12:59:34.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:35.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:59:35.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:35.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103170 -lt 249108103170 2026-02-19T12:59:35.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:35.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-219043332100 2026-02-19T12:59:35.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:35.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:59:35.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-219043332100 2026-02-19T12:59:35.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:35.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=219043332100 2026-02-19T12:59:35.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 219043332100' 2026-02-19T12:59:35.231 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 219043332100 2026-02-19T12:59:35.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:59:35.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 219043332100 -lt 219043332100 2026-02-19T12:59:35.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:35.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542157 2026-02-19T12:59:35.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:35.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:59:35.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542157 2026-02-19T12:59:35.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:35.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542157 2026-02-19T12:59:35.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542157' 2026-02-19T12:59:35.396 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542157 2026-02-19T12:59:35.397 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:59:35.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542157 -lt 60129542157 2026-02-19T12:59:35.562 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:59:35.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:35.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:35.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T12:59:35.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:59:35.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:59:35.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:59:35.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:59:35.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:59:35.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:59:35.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:59:35.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T12:59:35.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:59:35.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:35.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:36.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T12:59:36.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:59:36.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:59:36.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3784: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-19T12:59:36.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:59:36.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:59:36.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T12:59:36.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:59:36.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-19T12:59:36.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:59:36.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:59:36.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T12:59:36.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:59:36.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-19T12:59:36.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:59:36.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:59:36.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:59:36.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:59:36.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:59:36.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:59:36.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-19T12:59:36.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:59:36.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:59:36.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T12:59:36.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:59:36.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:59:36.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-19T12:59:36.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-19T12:59:36.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:59:36.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:59:36.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T12:59:36.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:59:36.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:59:36.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:59:36.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:59:36.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:59:36.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:59:36.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:59:36.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:59:36.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:59:36.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:59:36.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:59:36.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:59:36.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:59:36.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:59:36.742 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:59:36.742 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:59:36.742 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:59:36.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:36.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:36.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:59:36.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:59:36.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:59:36.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:59:36.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:59:36.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:59:36.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:59:36.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:59:36.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:59:36.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:59:36.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T12:59:36.745 INFO:tasks.workunit.client.0.trial144.stderr:start osd.2 2026-02-19T12:59:36.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:59:36.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-19T12:59:36.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T12:59:36.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:59:36.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:59:36.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:59:36.762 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:36.757+0000 7f6678a8fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:36.763 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:36.761+0000 7f6678a8fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:36.764 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:36.761+0000 7f6678a8fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:36.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T12:59:36.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:59:36.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:59:36.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:59:36.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:59:36.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:36.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:59:36.909 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:59:36.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:36.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:59:37.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:59:37.136 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:37.133+0000 7f6678a8fa40 -1 Falling back to public interface 2026-02-19T12:59:37.769 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:37.765+0000 7f6678a8fa40 -1 osd.2 60 log_to_monitors true 2026-02-19T12:59:38.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:59:38.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:38.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:59:38.067 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:38.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:38.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:59:38.223 INFO:tasks.workunit.client.0.trial144.stderr:osd.2 up in weight 1 up_from 63 up_thru 63 down_at 60 last_clean_interval [14,60) [v2:127.0.0.1:6818/1969158735,v1:127.0.0.1:6819/1969158735] [v2:127.0.0.1:6820/1969158735,v1:127.0.0.1:6821/1969158735] exists,up 61dd9924-18ad-468d-a81d-ab2d915ddbd0 2026-02-19T12:59:38.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:59:38.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:59:38.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:59:38.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:59:38.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:59:38.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:59:38.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:59:38.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:59:38.224 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:59:38.225 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:59:38.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:59:38.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:59:38.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:59:38.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:59:38.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:59:38.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:59:38.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:59:38.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:59:38.331 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:59:38.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:59:38.496 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:38.496 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:59:38.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:59:38.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:38.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:59:38.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=249108103172 2026-02-19T12:59:38.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 249108103172 2026-02-19T12:59:38.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103172' 2026-02-19T12:59:38.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:38.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:59:38.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332102 2026-02-19T12:59:38.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332102 2026-02-19T12:59:38.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103172 1-219043332102' 2026-02-19T12:59:38.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:38.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:59:38.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=270582939650 2026-02-19T12:59:38.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 270582939650 2026-02-19T12:59:38.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103172 1-219043332102 2-270582939650' 2026-02-19T12:59:38.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:38.714 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-249108103172 2026-02-19T12:59:38.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:38.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:59:38.716 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-249108103172 2026-02-19T12:59:38.716 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:38.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=249108103172 2026-02-19T12:59:38.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 249108103172' 2026-02-19T12:59:38.718 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 249108103172 2026-02-19T12:59:38.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:38.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103171 -lt 249108103172 2026-02-19T12:59:38.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:39.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:59:39.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:40.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103171 -lt 249108103172 2026-02-19T12:59:40.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:41.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:59:41.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:41.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103172 -lt 249108103172 2026-02-19T12:59:41.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:41.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-219043332102 2026-02-19T12:59:41.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:41.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:59:41.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-219043332102 2026-02-19T12:59:41.214 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:41.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=219043332102 2026-02-19T12:59:41.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 219043332102' 2026-02-19T12:59:41.215 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 219043332102 2026-02-19T12:59:41.216 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:59:41.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 219043332102 -lt 219043332102 2026-02-19T12:59:41.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:41.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-270582939650 2026-02-19T12:59:41.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:41.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:59:41.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-270582939650 2026-02-19T12:59:41.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:41.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=270582939650 2026-02-19T12:59:41.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 270582939650' 2026-02-19T12:59:41.379 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 270582939650 2026-02-19T12:59:41.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:59:41.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 270582939650 -lt 270582939650 2026-02-19T12:59:41.539 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:59:41.539 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:41.539 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:41.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T12:59:41.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:59:41.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:59:41.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:59:41.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:59:41.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:59:41.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:59:41.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:59:41.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T12:59:41.890 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:59:41.890 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:41.890 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:42.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T12:59:42.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:59:42.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:59:42.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3785: corrupt_scrub_erasure: echo -n val3-EOBJ4 2026-02-19T12:59:42.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3786: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-19T12:59:42.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:59:42.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:59:42.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T12:59:42.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:59:42.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-19T12:59:42.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:59:42.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:59:42.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T12:59:42.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:59:42.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-19T12:59:42.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:59:42.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:59:42.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:59:42.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:59:42.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:59:42.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:59:42.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-19T12:59:42.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:59:42.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:59:42.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T12:59:42.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:59:42.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:59:42.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-19T12:59:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-19T12:59:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:59:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:59:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T12:59:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:59:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T12:59:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:59:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:59:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:59:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:59:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T12:59:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T12:59:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:59:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:59:42.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:59:42.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:59:42.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:59:42.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:59:42.668 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:59:42.668 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:59:42.668 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:59:42.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:42.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:42.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:59:42.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:59:42.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:59:42.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:59:42.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:59:42.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:59:42.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:59:42.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:59:42.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:59:42.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T12:59:42.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T12:59:42.670 INFO:tasks.workunit.client.0.trial144.stderr:start osd.2 2026-02-19T12:59:42.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:59:42.671 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-19T12:59:42.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T12:59:42.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:59:42.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:59:42.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:59:42.690 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:42.685+0000 7fd74333da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:42.691 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:42.689+0000 7fd74333da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:42.692 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:42.689+0000 7fd74333da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:42.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T12:59:42.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:59:42.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T12:59:42.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:59:42.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:59:42.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:42.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:59:42.832 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:59:42.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:42.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:59:42.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:59:43.089 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:43.085+0000 7fd74333da40 -1 Falling back to public interface 2026-02-19T12:59:43.659 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:43.657+0000 7fd74333da40 -1 osd.2 65 log_to_monitors true 2026-02-19T12:59:43.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:59:43.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:43.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:59:43.993 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:43.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:43.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T12:59:44.146 INFO:tasks.workunit.client.0.trial144.stderr:osd.2 up in weight 1 up_from 69 up_thru 69 down_at 66 last_clean_interval [63,65) [v2:127.0.0.1:6818/466478485,v1:127.0.0.1:6819/466478485] [v2:127.0.0.1:6820/466478485,v1:127.0.0.1:6821/466478485] exists,up 61dd9924-18ad-468d-a81d-ab2d915ddbd0 2026-02-19T12:59:44.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:59:44.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:59:44.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:59:44.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:59:44.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:59:44.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:59:44.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:59:44.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:59:44.148 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:59:44.148 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:59:44.148 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:59:44.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:59:44.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:59:44.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:59:44.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:59:44.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:59:44.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:59:44.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:59:44.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:59:44.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:59:44.427 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:44.427 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:59:44.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:59:44.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:44.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:59:44.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=249108103174 2026-02-19T12:59:44.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 249108103174 2026-02-19T12:59:44.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103174' 2026-02-19T12:59:44.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:44.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:59:44.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332104 2026-02-19T12:59:44.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332104 2026-02-19T12:59:44.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103174 1-219043332104' 2026-02-19T12:59:44.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:44.571 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:59:44.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743426 2026-02-19T12:59:44.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743426 2026-02-19T12:59:44.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103174 1-219043332104 2-296352743426' 2026-02-19T12:59:44.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:44.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-249108103174 2026-02-19T12:59:44.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:44.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:59:44.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-249108103174 2026-02-19T12:59:44.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:44.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=249108103174 2026-02-19T12:59:44.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 249108103174' 2026-02-19T12:59:44.646 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 249108103174 2026-02-19T12:59:44.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:44.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103173 -lt 249108103174 2026-02-19T12:59:44.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:45.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:59:45.816 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:45.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103173 -lt 249108103174 2026-02-19T12:59:45.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:46.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:59:46.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:47.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103174 -lt 249108103174 2026-02-19T12:59:47.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:47.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-219043332104 2026-02-19T12:59:47.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:47.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:59:47.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-219043332104 2026-02-19T12:59:47.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:47.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=219043332104 2026-02-19T12:59:47.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 219043332104' 2026-02-19T12:59:47.142 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 219043332104 2026-02-19T12:59:47.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:59:47.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 219043332105 -lt 219043332104 2026-02-19T12:59:47.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:47.308 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743426 2026-02-19T12:59:47.308 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:47.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:59:47.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743426 2026-02-19T12:59:47.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:47.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743426 2026-02-19T12:59:47.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743426' 2026-02-19T12:59:47.312 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 296352743426 2026-02-19T12:59:47.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:59:47.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743426 -lt 296352743426 2026-02-19T12:59:47.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:59:47.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:47.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:47.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T12:59:47.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:59:47.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:59:47.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:59:47.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:59:47.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:59:47.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:59:47.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:59:47.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T12:59:47.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:59:47.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:47.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:48.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T12:59:48.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:59:48.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:59:48.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3787: corrupt_scrub_erasure: rm td/osd-scrub-repair/bad-val td/osd-scrub-repair/newval 2026-02-19T12:59:48.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-19T12:59:48.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3753: corrupt_scrub_erasure: objname=EOBJ5 2026-02-19T12:59:48.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3754: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ5 2026-02-19T12:59:48.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T12:59:48.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T12:59:48.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ5 2026-02-19T12:59:48.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T12:59:48.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T12:59:48.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T12:59:48.273 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T12:59:48.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T12:59:48.475 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T12:59:48.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T12:59:48.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T12:59:48.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ5 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:59:48.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: expr 5 % 2 2026-02-19T12:59:48.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: local osd=1 2026-02-19T12:59:48.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3758: corrupt_scrub_erasure: case $i in 2026-02-19T12:59:48.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3792: corrupt_scrub_erasure: dd if=/dev/urandom of=td/osd-scrub-repair/CORRUPT bs=2048 count=2 2026-02-19T12:59:48.511 INFO:tasks.workunit.client.0.trial144.stderr:2+0 records in 2026-02-19T12:59:48.512 INFO:tasks.workunit.client.0.trial144.stderr:2+0 records out 2026-02-19T12:59:48.512 INFO:tasks.workunit.client.0.trial144.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000100225 s, 40.9 MB/s 2026-02-19T12:59:48.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3793: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T12:59:48.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:59:48.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:59:48.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:59:48.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:59:48.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T12:59:48.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:59:48.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:59:48.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:59:48.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:59:48.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:59:48.513 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:59:48.513 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:59:48.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:59:48.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:59:48.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:59:48.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:59:48.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T12:59:48.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:59:48.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:59:48.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:59:48.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:59:48.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:59:48.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T12:59:49.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T12:59:49.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:59:49.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:59:49.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T12:59:49.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:59:49.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:59:49.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:59:49.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:59:49.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:59:49.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:59:49.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T12:59:49.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T12:59:49.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:59:49.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:59:49.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:59:49.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:59:49.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:59:49.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:59:49.108 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:59:49.108 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:59:49.108 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:59:49.109 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:49.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:49.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:59:49.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:59:49.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:59:49.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:59:49.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:59:49.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:59:49.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:59:49.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:59:49.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:59:49.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T12:59:49.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T12:59:49.110 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T12:59:49.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:59:49.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T12:59:49.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T12:59:49.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:59:49.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:59:49.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:59:49.130 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:49.125+0000 7f0dad807a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:49.131 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:49.129+0000 7f0dad807a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:49.132 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:49.129+0000 7f0dad807a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:49.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T12:59:49.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:59:49.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T12:59:49.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:59:49.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:59:49.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:49.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:59:49.276 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:59:49.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:49.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:59:49.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:59:49.560 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:49.557+0000 7f0dad807a40 -1 Falling back to public interface 2026-02-19T12:59:50.108 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:50.105+0000 7f0dad807a40 -1 osd.1 72 log_to_monitors true 2026-02-19T12:59:50.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:59:50.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:50.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:59:50.433 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:50.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:50.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T12:59:50.587 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 76 up_thru 76 down_at 73 last_clean_interval [51,72) [v2:127.0.0.1:6810/895987730,v1:127.0.0.1:6811/895987730] [v2:127.0.0.1:6812/895987730,v1:127.0.0.1:6813/895987730] exists,up 063d83eb-ff81-4f56-b415-8fbd1f91414f 2026-02-19T12:59:50.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:59:50.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:59:50.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:59:50.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:59:50.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:59:50.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:59:50.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:59:50.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:59:50.589 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:59:50.589 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:59:50.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:59:50.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:59:50.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:59:50.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:59:50.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:59:50.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:59:50.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:59:50.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:59:50.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:59:50.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:59:50.867 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:50.867 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:59:50.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:59:50.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:50.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:59:50.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=249108103176 2026-02-19T12:59:50.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 249108103176 2026-02-19T12:59:50.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103176' 2026-02-19T12:59:50.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:50.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:59:51.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=326417514498 2026-02-19T12:59:51.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 326417514498 2026-02-19T12:59:51.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103176 1-326417514498' 2026-02-19T12:59:51.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:51.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:59:51.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743428 2026-02-19T12:59:51.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743428 2026-02-19T12:59:51.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103176 1-326417514498 2-296352743428' 2026-02-19T12:59:51.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:51.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-249108103176 2026-02-19T12:59:51.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:51.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:59:51.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-249108103176 2026-02-19T12:59:51.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:51.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=249108103176 2026-02-19T12:59:51.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 249108103176' 2026-02-19T12:59:51.088 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 249108103176 2026-02-19T12:59:51.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:51.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103175 -lt 249108103176 2026-02-19T12:59:51.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:52.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:59:52.255 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:52.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103177 -lt 249108103176 2026-02-19T12:59:52.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:52.417 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-326417514498 2026-02-19T12:59:52.417 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:52.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:59:52.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-326417514498 2026-02-19T12:59:52.420 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:52.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=326417514498 2026-02-19T12:59:52.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 326417514498' 2026-02-19T12:59:52.421 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 326417514498 2026-02-19T12:59:52.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:59:52.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 326417514498 -lt 326417514498 2026-02-19T12:59:52.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:52.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743428 2026-02-19T12:59:52.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:52.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:59:52.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743428 2026-02-19T12:59:52.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:52.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743428 2026-02-19T12:59:52.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743428' 2026-02-19T12:59:52.595 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 296352743428 2026-02-19T12:59:52.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:59:52.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743428 -lt 296352743428 2026-02-19T12:59:52.756 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:59:52.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:52.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:52.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T12:59:52.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:59:52.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:59:52.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:59:52.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:59:52.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:59:52.952 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:59:52.952 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:59:53.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T12:59:53.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:59:53.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:53.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:53.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T12:59:53.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:59:53.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:59:53.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-19T12:59:53.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3753: corrupt_scrub_erasure: objname=EOBJ6 2026-02-19T12:59:53.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3754: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ6 2026-02-19T12:59:53.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T12:59:53.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T12:59:53.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ6 2026-02-19T12:59:53.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T12:59:53.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T12:59:53.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T12:59:53.508 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T12:59:53.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T12:59:53.711 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T12:59:53.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T12:59:53.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T12:59:53.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ6 td/osd-scrub-repair/ORIGINAL 2026-02-19T12:59:53.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: expr 6 % 2 2026-02-19T12:59:53.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: local osd=0 2026-02-19T12:59:53.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3758: corrupt_scrub_erasure: case $i in 2026-02-19T12:59:53.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3797: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-19T12:59:53.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:59:53.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:59:53.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T12:59:53.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:59:53.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-19T12:59:53.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:59:53.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:59:53.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T12:59:53.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:59:53.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T12:59:53.755 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:59:53.755 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:59:53.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:59:53.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:59:53.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:59:54.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:59:54.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-19T12:59:54.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:59:54.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:59:54.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T12:59:54.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:59:54.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:59:54.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ6 rm-attr hinfo_key 2026-02-19T12:59:54.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T12:59:54.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T12:59:54.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T12:59:54.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T12:59:54.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T12:59:54.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T12:59:54.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T12:59:54.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T12:59:54.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T12:59:54.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T12:59:54.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T12:59:54.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T12:59:54.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T12:59:54.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T12:59:54.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T12:59:54.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T12:59:54.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T12:59:54.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T12:59:54.344 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T12:59:54.344 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T12:59:54.344 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T12:59:54.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:54.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T12:59:54.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T12:59:54.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T12:59:54.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T12:59:54.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T12:59:54.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T12:59:54.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T12:59:54.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T12:59:54.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T12:59:54.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T12:59:54.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T12:59:54.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T12:59:54.346 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T12:59:54.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T12:59:54.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T12:59:54.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T12:59:54.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T12:59:54.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T12:59:54.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T12:59:54.364 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:54.361+0000 7f260f8a2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:54.366 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:54.361+0000 7f260f8a2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:54.366 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:54.365+0000 7f260f8a2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T12:59:54.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T12:59:54.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T12:59:54.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T12:59:54.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T12:59:54.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T12:59:54.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:54.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T12:59:54.510 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T12:59:54.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:54.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:59:54.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T12:59:54.728 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:54.725+0000 7f260f8a2a40 -1 Falling back to public interface 2026-02-19T12:59:55.346 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T12:59:55.341+0000 7f260f8a2a40 -1 osd.0 79 log_to_monitors true 2026-02-19T12:59:55.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T12:59:55.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T12:59:55.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T12:59:55.670 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:55.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T12:59:55.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T12:59:55.827 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 83 up_thru 83 down_at 80 last_clean_interval [58,79) [v2:127.0.0.1:6802/353605858,v1:127.0.0.1:6803/353605858] [v2:127.0.0.1:6804/353605858,v1:127.0.0.1:6805/353605858] exists,up a97b81ca-b503-46b6-9623-80a650d2efd1 2026-02-19T12:59:55.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T12:59:55.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T12:59:55.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T12:59:55.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T12:59:55.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T12:59:55.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T12:59:55.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T12:59:55.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T12:59:55.829 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T12:59:55.829 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T12:59:55.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T12:59:55.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T12:59:55.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T12:59:55.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T12:59:55.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T12:59:55.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T12:59:55.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T12:59:55.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T12:59:55.937 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T12:59:56.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T12:59:56.096 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T12:59:56.096 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T12:59:56.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T12:59:56.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:56.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T12:59:56.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285570 2026-02-19T12:59:56.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285570 2026-02-19T12:59:56.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-356482285570' 2026-02-19T12:59:56.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:56.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T12:59:56.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=326417514500 2026-02-19T12:59:56.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 326417514500 2026-02-19T12:59:56.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-356482285570 1-326417514500' 2026-02-19T12:59:56.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T12:59:56.245 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T12:59:56.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743430 2026-02-19T12:59:56.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743430 2026-02-19T12:59:56.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-356482285570 1-326417514500 2-296352743430' 2026-02-19T12:59:56.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:56.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-356482285570 2026-02-19T12:59:56.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:56.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T12:59:56.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-356482285570 2026-02-19T12:59:56.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:56.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285570 2026-02-19T12:59:56.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 356482285570' 2026-02-19T12:59:56.321 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 356482285570 2026-02-19T12:59:56.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:56.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 356482285570 2026-02-19T12:59:56.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:57.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T12:59:57.488 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:57.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 356482285570 2026-02-19T12:59:57.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T12:59:58.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T12:59:58.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T12:59:58.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285570 -lt 356482285570 2026-02-19T12:59:58.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:58.807 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-326417514500 2026-02-19T12:59:58.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:58.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T12:59:58.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-326417514500 2026-02-19T12:59:58.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:58.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=326417514500 2026-02-19T12:59:58.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 326417514500' 2026-02-19T12:59:58.811 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 326417514500 2026-02-19T12:59:58.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T12:59:58.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 326417514500 -lt 326417514500 2026-02-19T12:59:58.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T12:59:58.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743430 2026-02-19T12:59:58.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T12:59:58.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T12:59:58.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743430 2026-02-19T12:59:58.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T12:59:58.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743430 2026-02-19T12:59:58.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743430' 2026-02-19T12:59:58.975 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 296352743430 2026-02-19T12:59:58.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T12:59:59.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743430 -lt 296352743430 2026-02-19T12:59:59.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T12:59:59.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:59.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:59.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T12:59:59.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T12:59:59.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T12:59:59.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T12:59:59.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T12:59:59.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T12:59:59.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T12:59:59.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T12:59:59.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T12:59:59.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T12:59:59.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T12:59:59.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T12:59:59.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T12:59:59.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T12:59:59.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T12:59:59.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3798: corrupt_scrub_erasure: echo -n bad-val 2026-02-19T12:59:59.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3799: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-19T12:59:59.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T12:59:59.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T12:59:59.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T12:59:59.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T12:59:59.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-19T12:59:59.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T12:59:59.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T12:59:59.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T12:59:59.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T12:59:59.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T12:59:59.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T12:59:59.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T12:59:59.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T12:59:59.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T12:59:59.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T12:59:59.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T12:59:59.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-19T12:59:59.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T12:59:59.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T12:59:59.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T12:59:59.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T12:59:59.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T12:59:59.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-19T13:00:00.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:00:00.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:00:00.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:00:00.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:00:00.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:00:00.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:00:00.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:00:00.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:00:00.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:00:00.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:00:00.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:00:00.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:00:00.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:00:00.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:00:00.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:00:00.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:00:00.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:00:00.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:00:00.278 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:00:00.278 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:00:00.278 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:00:00.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:00:00.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:00:00.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:00:00.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:00:00.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:00:00.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:00:00.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:00:00.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:00:00.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:00:00.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:00:00.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:00:00.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:00:00.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:00:00.281 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:00:00.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:00:00.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:00:00.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:00:00.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:00:00.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:00:00.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:00:00.292 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:00.288+0000 7fb1da940a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:00.294 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:00.288+0000 7fb1da940a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:00.295 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:00.292+0000 7fb1da940a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:00.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:00:00.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:00:00.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:00:00.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:00:00.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:00:00.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:00:00.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:00:00.436 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:00:00.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:00:00.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:00:00.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:00:00.696 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:00.692+0000 7fb1da940a40 -1 Falling back to public interface 2026-02-19T13:00:01.331 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:01.328+0000 7fb1da940a40 -1 osd.1 85 log_to_monitors true 2026-02-19T13:00:01.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:00:01.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:00:01.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:00:01.594 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:00:01.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:00:01.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:00:01.755 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 89 up_thru 89 down_at 86 last_clean_interval [76,85) [v2:127.0.0.1:6810/1520390302,v1:127.0.0.1:6811/1520390302] [v2:127.0.0.1:6812/1520390302,v1:127.0.0.1:6813/1520390302] exists,up 063d83eb-ff81-4f56-b415-8fbd1f91414f 2026-02-19T13:00:01.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:00:01.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:00:01.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:00:01.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:00:01.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:00:01.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:00:01.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:00:01.755 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:00:01.756 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:00:01.756 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:00:01.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:00:01.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:00:01.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:00:01.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:00:01.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:00:01.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:00:01.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:00:01.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:00:01.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:00:02.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:00:02.029 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:00:02.029 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:00:02.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:00:02.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:02.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:00:02.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285572 2026-02-19T13:00:02.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285572 2026-02-19T13:00:02.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-356482285572' 2026-02-19T13:00:02.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:02.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:00:02.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=382252089346 2026-02-19T13:00:02.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 382252089346 2026-02-19T13:00:02.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-356482285572 1-382252089346' 2026-02-19T13:00:02.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:02.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:00:02.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743432 2026-02-19T13:00:02.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743432 2026-02-19T13:00:02.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-356482285572 1-382252089346 2-296352743432' 2026-02-19T13:00:02.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:02.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-356482285572 2026-02-19T13:00:02.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:02.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:00:02.249 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-356482285572 2026-02-19T13:00:02.249 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:02.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285572 2026-02-19T13:00:02.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 356482285572' 2026-02-19T13:00:02.251 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 356482285572 2026-02-19T13:00:02.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:02.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285571 -lt 356482285572 2026-02-19T13:00:02.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:00:03.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:00:03.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:03.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285571 -lt 356482285572 2026-02-19T13:00:03.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:00:04.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:00:04.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:04.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285572 -lt 356482285572 2026-02-19T13:00:04.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:04.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-382252089346 2026-02-19T13:00:04.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:04.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:00:04.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-382252089346 2026-02-19T13:00:04.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:04.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=382252089346 2026-02-19T13:00:04.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 382252089346' 2026-02-19T13:00:04.737 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 382252089346 2026-02-19T13:00:04.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:00:04.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 382252089346 -lt 382252089346 2026-02-19T13:00:04.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:04.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743432 2026-02-19T13:00:04.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:04.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:00:04.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743432 2026-02-19T13:00:04.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:04.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743432 2026-02-19T13:00:04.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743432' 2026-02-19T13:00:04.915 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 296352743432 2026-02-19T13:00:04.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:00:05.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743433 -lt 296352743432 2026-02-19T13:00:05.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:00:05.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:00:05.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:00:05.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:00:05.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:00:05.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:00:05.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:00:05.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:00:05.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:00:05.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:00:05.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:00:05.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:00:05.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:00:05.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:00:05.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:00:05.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:00:05.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:00:05.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:00:05.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-19T13:00:05.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3753: corrupt_scrub_erasure: objname=EOBJ7 2026-02-19T13:00:05.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3754: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ7 2026-02-19T13:00:05.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:00:05.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T13:00:05.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ7 2026-02-19T13:00:05.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:00:05.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:00:05.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:00:05.823 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:00:05.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:00:06.028 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:00:06.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:00:06.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:00:06.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ7 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:00:06.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: expr 7 % 2 2026-02-19T13:00:06.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: local osd=1 2026-02-19T13:00:06.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3758: corrupt_scrub_erasure: case $i in 2026-02-19T13:00:06.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3803: corrupt_scrub_erasure: local payload=MAKETHISDIFFERENTFROMOTHEROBJECTS 2026-02-19T13:00:06.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3804: corrupt_scrub_erasure: echo MAKETHISDIFFERENTFROMOTHEROBJECTS 2026-02-19T13:00:06.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3805: corrupt_scrub_erasure: rados --pool ecpool put EOBJ7 td/osd-scrub-repair/DIFFERENT 2026-02-19T13:00:06.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3808: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-19T13:00:06.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:00:06.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:00:06.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:00:06.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:00:06.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-19T13:00:06.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:00:06.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:00:06.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:00:06.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:00:06.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:00:06.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:00:06.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:00:06.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:00:06.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:00:06.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:00:06.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:00:06.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-19T13:00:06.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:00:06.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:00:06.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:00:06.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:00:06.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:00:06.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ1 get-attr hinfo_key 2026-02-19T13:00:06.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:00:06.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:00:06.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:00:06.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:00:06.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:00:06.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:00:06.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:00:06.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:00:06.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:00:06.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:00:06.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:00:06.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:00:06.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:00:06.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:00:06.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:00:06.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:00:06.552 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:00:06.552 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:00:06.552 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:00:06.552 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:00:06.552 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:00:06.553 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:00:06.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:00:06.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:00:06.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:00:06.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:00:06.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:00:06.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:00:06.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:00:06.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:00:06.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:00:06.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:00:06.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:00:06.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:00:06.554 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:00:06.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:00:06.554 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:00:06.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:00:06.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:00:06.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:00:06.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:00:06.573 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:06.568+0000 7fb878c73a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:06.575 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:06.572+0000 7fb878c73a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:06.576 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:06.572+0000 7fb878c73a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:06.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:00:06.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:00:06.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:00:06.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:00:06.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:00:06.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:00:06.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:00:06.711 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:00:06.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:00:06.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:00:06.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:00:06.952 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:06.948+0000 7fb878c73a40 -1 Falling back to public interface 2026-02-19T13:00:07.603 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:07.600+0000 7fb878c73a40 -1 osd.0 92 log_to_monitors true 2026-02-19T13:00:07.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:00:07.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:00:07.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:00:07.876 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:00:07.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:00:07.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:00:08.031 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 96 up_thru 96 down_at 93 last_clean_interval [83,92) [v2:127.0.0.1:6802/2172852573,v1:127.0.0.1:6803/2172852573] [v2:127.0.0.1:6804/2172852573,v1:127.0.0.1:6805/2172852573] exists,up a97b81ca-b503-46b6-9623-80a650d2efd1 2026-02-19T13:00:08.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:00:08.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:00:08.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:00:08.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:00:08.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:00:08.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:00:08.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:00:08.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:00:08.032 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:00:08.032 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:00:08.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:00:08.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:00:08.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:00:08.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:00:08.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:00:08.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:00:08.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:00:08.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:00:08.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:00:08.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:00:08.303 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:00:08.303 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:00:08.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:00:08.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:08.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:00:08.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=412316860418 2026-02-19T13:00:08.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 412316860418 2026-02-19T13:00:08.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-412316860418' 2026-02-19T13:00:08.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:08.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:00:08.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=382252089348 2026-02-19T13:00:08.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 382252089348 2026-02-19T13:00:08.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-412316860418 1-382252089348' 2026-02-19T13:00:08.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:08.448 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:00:08.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743434 2026-02-19T13:00:08.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743434 2026-02-19T13:00:08.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-412316860418 1-382252089348 2-296352743434' 2026-02-19T13:00:08.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:08.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-412316860418 2026-02-19T13:00:08.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:08.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:00:08.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-412316860418 2026-02-19T13:00:08.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:08.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=412316860418 2026-02-19T13:00:08.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 412316860418' 2026-02-19T13:00:08.525 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 412316860418 2026-02-19T13:00:08.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:08.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 412316860418 2026-02-19T13:00:08.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:00:09.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:00:09.691 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:09.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 412316860418 2026-02-19T13:00:09.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:00:10.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:00:10.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:11.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 412316860418 -lt 412316860418 2026-02-19T13:00:11.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:11.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-382252089348 2026-02-19T13:00:11.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:11.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:00:11.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-382252089348 2026-02-19T13:00:11.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:11.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=382252089348 2026-02-19T13:00:11.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 382252089348' 2026-02-19T13:00:11.017 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 382252089348 2026-02-19T13:00:11.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:00:11.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 382252089348 -lt 382252089348 2026-02-19T13:00:11.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:11.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743434 2026-02-19T13:00:11.185 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:11.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:00:11.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743434 2026-02-19T13:00:11.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:11.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743434 2026-02-19T13:00:11.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743434' 2026-02-19T13:00:11.188 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 296352743434 2026-02-19T13:00:11.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:00:11.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743435 -lt 296352743434 2026-02-19T13:00:11.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:00:11.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:00:11.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:00:11.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:00:11.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:00:11.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:00:11.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:00:11.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:00:11.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:00:11.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:00:11.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:00:11.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:00:11.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:00:11.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:00:11.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:00:11.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:00:11.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:00:11.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:00:11.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3809: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-19T13:00:11.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:00:11.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:00:11.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:00:11.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:00:11.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-19T13:00:11.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:00:11.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:00:11.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:00:11.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:00:11.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:00:11.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:00:11.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:00:11.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:00:11.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:00:11.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:00:12.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:00:12.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-19T13:00:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:00:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:00:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:00:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:00:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:00:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-19T13:00:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:00:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:00:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:00:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:00:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:00:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:00:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:00:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:00:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:00:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:00:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:00:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:00:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:00:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:00:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:00:12.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:00:12.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:00:12.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:00:12.456 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:00:12.456 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:00:12.456 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:00:12.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:00:12.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:00:12.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:00:12.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:00:12.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:00:12.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:00:12.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:00:12.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:00:12.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:00:12.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:00:12.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:00:12.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:00:12.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:00:12.458 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:00:12.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:00:12.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:00:12.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:00:12.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:00:12.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:00:12.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:00:12.476 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:12.472+0000 7f301aa2ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:12.478 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:12.472+0000 7f301aa2ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:12.479 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:12.476+0000 7f301aa2ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:12.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:00:12.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:00:12.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:00:12.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:00:12.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:00:12.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:00:12.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:00:12.620 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:00:12.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:00:12.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:00:12.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:00:12.908 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:12.904+0000 7f301aa2ca40 -1 Falling back to public interface 2026-02-19T13:00:13.445 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:13.440+0000 7f301aa2ca40 -1 osd.0 97 log_to_monitors true 2026-02-19T13:00:13.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:00:13.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:00:13.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:00:13.782 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:00:13.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:00:13.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:00:13.943 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 101 up_thru 101 down_at 98 last_clean_interval [96,97) [v2:127.0.0.1:6802/2021801898,v1:127.0.0.1:6803/2021801898] [v2:127.0.0.1:6804/2021801898,v1:127.0.0.1:6805/2021801898] exists,up a97b81ca-b503-46b6-9623-80a650d2efd1 2026-02-19T13:00:13.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:00:13.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:00:13.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:00:13.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:00:13.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:00:13.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:00:13.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:00:13.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:00:13.945 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:00:13.945 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:00:13.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:00:13.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:00:13.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:00:14.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:00:14.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:00:14.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:00:14.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:00:14.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:00:14.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:00:14.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:00:14.223 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:00:14.223 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:00:14.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:00:14.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:14.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:00:14.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=433791696898 2026-02-19T13:00:14.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 433791696898 2026-02-19T13:00:14.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-433791696898' 2026-02-19T13:00:14.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:14.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:00:14.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=382252089350 2026-02-19T13:00:14.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 382252089350 2026-02-19T13:00:14.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-433791696898 1-382252089350' 2026-02-19T13:00:14.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:14.369 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:00:14.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743437 2026-02-19T13:00:14.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743437 2026-02-19T13:00:14.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-433791696898 1-382252089350 2-296352743437' 2026-02-19T13:00:14.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:14.444 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-433791696898 2026-02-19T13:00:14.444 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:14.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:00:14.446 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-433791696898 2026-02-19T13:00:14.446 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:14.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=433791696898 2026-02-19T13:00:14.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 433791696898' 2026-02-19T13:00:14.448 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 433791696898 2026-02-19T13:00:14.448 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:14.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 433791696898 2026-02-19T13:00:14.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:00:15.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:00:15.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:15.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 433791696898 2026-02-19T13:00:15.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:00:16.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:00:16.774 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:16.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 433791696898 -lt 433791696898 2026-02-19T13:00:16.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:16.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-382252089350 2026-02-19T13:00:16.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:16.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:00:16.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-382252089350 2026-02-19T13:00:16.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:16.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=382252089350 2026-02-19T13:00:16.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 382252089350' 2026-02-19T13:00:16.937 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 382252089350 2026-02-19T13:00:16.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:00:17.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 382252089350 -lt 382252089350 2026-02-19T13:00:17.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:17.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743437 2026-02-19T13:00:17.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:17.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:00:17.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743437 2026-02-19T13:00:17.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:17.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743437 2026-02-19T13:00:17.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743437' 2026-02-19T13:00:17.116 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 296352743437 2026-02-19T13:00:17.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:00:17.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743437 -lt 296352743437 2026-02-19T13:00:17.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:00:17.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:00:17.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:00:17.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:00:17.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:00:17.463 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:00:17.463 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:00:17.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:00:17.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:00:17.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:00:17.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:00:17.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:00:17.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:00:17.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:00:17.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:00:17.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:00:17.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:00:17.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:00:17.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3810: corrupt_scrub_erasure: rm -f td/osd-scrub-repair/hinfo 2026-02-19T13:00:17.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3816: corrupt_scrub_erasure: get_pg ecpool EOBJ0 2026-02-19T13:00:17.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T13:00:17.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=EOBJ0 2026-02-19T13:00:17.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool EOBJ0 2026-02-19T13:00:17.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:00:17.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3816: corrupt_scrub_erasure: local pg=3.0 2026-02-19T13:00:17.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3818: corrupt_scrub_erasure: pg_scrub 3.0 2026-02-19T13:00:17.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=3.0 2026-02-19T13:00:17.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 3.0 2026-02-19T13:00:17.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=3.0 2026-02-19T13:00:17.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:00:17.976 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:00:17.976 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:00:17.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:00:17.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:00:17.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:00:18.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:00:18.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:00:18.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:00:18.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:00:18.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:00:18.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:00:18.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:00:18.431 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:00:18.431 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:00:18.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:00:18.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:18.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:00:18.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=433791696899 2026-02-19T13:00:18.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 433791696899 2026-02-19T13:00:18.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-433791696899' 2026-02-19T13:00:18.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:18.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:00:18.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=382252089352 2026-02-19T13:00:18.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 382252089352 2026-02-19T13:00:18.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-433791696899 1-382252089352' 2026-02-19T13:00:18.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:18.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:00:18.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743438 2026-02-19T13:00:18.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743438 2026-02-19T13:00:18.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-433791696899 1-382252089352 2-296352743438' 2026-02-19T13:00:18.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:18.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-433791696899 2026-02-19T13:00:18.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:18.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:00:18.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-433791696899 2026-02-19T13:00:18.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:18.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=433791696899 2026-02-19T13:00:18.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 433791696899' 2026-02-19T13:00:18.654 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 433791696899 2026-02-19T13:00:18.654 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:18.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 433791696898 -lt 433791696899 2026-02-19T13:00:18.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:00:19.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:00:19.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:19.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 433791696898 -lt 433791696899 2026-02-19T13:00:19.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:00:20.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:00:20.984 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:21.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 433791696900 -lt 433791696899 2026-02-19T13:00:21.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:21.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-382252089352 2026-02-19T13:00:21.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:21.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:00:21.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-382252089352 2026-02-19T13:00:21.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:21.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=382252089352 2026-02-19T13:00:21.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 382252089352' 2026-02-19T13:00:21.150 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 382252089352 2026-02-19T13:00:21.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:00:21.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 382252089352 -lt 382252089352 2026-02-19T13:00:21.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:21.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743438 2026-02-19T13:00:21.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:21.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:00:21.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743438 2026-02-19T13:00:21.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:21.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743438 2026-02-19T13:00:21.315 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 296352743438 2026-02-19T13:00:21.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743438' 2026-02-19T13:00:21.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:00:21.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743439 -lt 296352743438 2026-02-19T13:00:21.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:00:21.471 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 3.0 loop 0 2026-02-19T13:00:21.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-19T13:00:21.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 3.0 2026-02-19T13:00:21.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=3.0 2026-02-19T13:00:21.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:00:21.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 3.0 query 2026-02-19T13:00:21.472 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:00:21.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-19T13:00:21.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:00:21.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:00:21.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:00:21.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 3.0 2026-02-19T13:00:21.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:21.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:00:21.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:21.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:00:21.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:21.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 3.0 2026-02-19T13:00:21.848 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0s0 on osd.1 to scrub 2026-02-19T13:00:21.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 3.0 2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:21.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:00:21.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:21.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:00:21.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:00:21.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:00:21.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:00:21.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:21.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:00:21.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:21.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:00:22.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:59:04.068459+0000 '>' 2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:22.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:00:23.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:00:23.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:00:23.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:00:23.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:23.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:00:23.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:23.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:00:23.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:59:04.068459+0000 '>' 2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:23.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:00:24.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:00:24.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:00:24.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:00:24.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:24.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:00:24.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:24.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:00:24.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:59:04.068459+0000 '>' 2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:24.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:00:25.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:00:25.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:00:25.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:00:25.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:25.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:00:25.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:25.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:00:25.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:59:04.068459+0000 '>' 2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:25.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:00:26.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:00:26.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:00:26.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:00:26.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:26.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:00:26.511 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:26.511 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:00:26.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:59:04.068459+0000 '>' 2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:26.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:00:27.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:00:27.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:00:27.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:00:27.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:27.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:00:27.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:27.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:00:27.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:59:04.068459+0000 '>' 2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:27.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:00:28.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:00:28.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:00:28.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:00:28.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:28.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:00:28.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:28.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:00:28.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:00:22.260876+0000 '>' 2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:28.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:00:28.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3820: corrupt_scrub_erasure: rados list-inconsistent-pg ecpool 2026-02-19T13:00:29.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3822: corrupt_scrub_erasure: jq '. | length' td/osd-scrub-repair/json 2026-02-19T13:00:29.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3822: corrupt_scrub_erasure: test 1 = 1 2026-02-19T13:00:29.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3824: corrupt_scrub_erasure: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-19T13:00:29.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3824: corrupt_scrub_erasure: test 3.0 = 3.0 2026-02-19T13:00:29.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3826: corrupt_scrub_erasure: rados list-inconsistent-obj 3.0 2026-02-19T13:00:29.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3828: corrupt_scrub_erasure: jq .epoch td/osd-scrub-repair/json 2026-02-19T13:00:29.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3828: corrupt_scrub_erasure: epoch=101 2026-02-19T13:00:29.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3830: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-19T13:00:29.071 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:00:29.071 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:00:29.071 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:00:29.071 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:00:29.071 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:00:29.071 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:00:29.071 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:00:29.071 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:00:29.071 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:00:29.071 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:00:29.071 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:00:29.071 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-19T13:00:29.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3830: corrupt_scrub_erasure: jq .inconsistents 2026-02-19T13:00:29.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3830: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:00:29.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4458: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-19T13:00:29.090 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:00:29.090 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:00:29.090 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:00:29.090 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:00:29.090 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:00:29.090 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:00:29.090 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:00:29.090 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:00:29.090 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:00:29.090 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:00:29.090 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:00:29.090 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-19T13:00:29.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4458: corrupt_scrub_erasure: jq .inconsistents 2026-02-19T13:00:29.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4458: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:00:29.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4459: corrupt_scrub_erasure: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-19T13:00:29.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-19T13:00:29.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4460: corrupt_scrub_erasure: test no = yes 2026-02-19T13:00:29.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4465: corrupt_scrub_erasure: test '' = yes 2026-02-19T13:00:29.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4470: corrupt_scrub_erasure: pg_deep_scrub 3.0 2026-02-19T13:00:29.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_deep_scrub: local pgid=3.0 2026-02-19T13:00:29.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_deep_scrub: wait_for_pg_clean 3.0 2026-02-19T13:00:29.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=3.0 2026-02-19T13:00:29.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:00:29.117 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:00:29.117 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:00:29.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:00:29.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:00:29.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:00:29.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:00:29.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:00:29.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:00:29.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:00:29.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:00:29.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:00:29.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:00:29.575 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:00:29.575 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:00:29.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:00:29.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:29.575 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:00:29.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=433791696903 2026-02-19T13:00:29.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 433791696903 2026-02-19T13:00:29.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-433791696903' 2026-02-19T13:00:29.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:29.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:00:29.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=382252089355 2026-02-19T13:00:29.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 382252089355 2026-02-19T13:00:29.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-433791696903 1-382252089355' 2026-02-19T13:00:29.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:29.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:00:29.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743442 2026-02-19T13:00:29.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743442 2026-02-19T13:00:29.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-433791696903 1-382252089355 2-296352743442' 2026-02-19T13:00:29.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:29.791 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-433791696903 2026-02-19T13:00:29.791 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:29.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:00:29.793 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-433791696903 2026-02-19T13:00:29.793 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:29.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=433791696903 2026-02-19T13:00:29.795 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 433791696903 2026-02-19T13:00:29.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 433791696903' 2026-02-19T13:00:29.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:29.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 433791696901 -lt 433791696903 2026-02-19T13:00:29.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:00:30.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:00:30.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:31.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 433791696903 -lt 433791696903 2026-02-19T13:00:31.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:31.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-382252089355 2026-02-19T13:00:31.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:31.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:00:31.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-382252089355 2026-02-19T13:00:31.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:31.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=382252089355 2026-02-19T13:00:31.118 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 382252089355 2026-02-19T13:00:31.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 382252089355' 2026-02-19T13:00:31.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:00:31.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 382252089355 -lt 382252089355 2026-02-19T13:00:31.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:31.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743442 2026-02-19T13:00:31.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:31.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:00:31.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743442 2026-02-19T13:00:31.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:31.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743442 2026-02-19T13:00:31.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743442' 2026-02-19T13:00:31.290 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 296352743442 2026-02-19T13:00:31.290 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:00:31.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743442 -lt 296352743442 2026-02-19T13:00:31.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:00:31.450 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 3.0 loop 0 2026-02-19T13:00:31.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-19T13:00:31.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 3.0 2026-02-19T13:00:31.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=3.0 2026-02-19T13:00:31.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:00:31.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 3.0 query 2026-02-19T13:00:31.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:00:31.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean+inconsistent 2026-02-19T13:00:31.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean+inconsistent == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:00:31.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:00:31.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:00:31.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:00:31.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:31.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:00:31.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:31.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:00:31.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: local last_scrub=2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:31.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_deep_scrub: ceph pg deep-scrub 3.0 2026-02-19T13:00:31.827 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0s0 on osd.1 to deep-scrub 2026-02-19T13:00:31.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_deep_scrub: wait_for_scrub 3.0 2026-02-19T12:59:04.068459+0000 last_deep_scrub_stamp 2026-02-19T13:00:31.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:00:31.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:31.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-19T13:00:31.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:00:31.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:00:31.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:00:31.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:31.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:00:31.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:31.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:00:31.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:59:04.068459+0000 '>' 2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:31.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:00:32.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:00:32.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:00:32.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:00:32.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:32.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:00:32.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:32.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:00:33.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:59:04.068459+0000 '>' 2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:33.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:00:34.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:00:34.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:00:34.153 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:00:34.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:34.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:00:34.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:34.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:00:34.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:59:04.068459+0000 '>' 2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:34.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:00:35.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:00:35.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:00:35.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:00:35.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:35.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:00:35.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:35.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:00:35.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:59:04.068459+0000 '>' 2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:35.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:00:36.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:00:36.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:00:36.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:00:36.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:36.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:00:36.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:36.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:00:36.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:59:04.068459+0000 '>' 2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:36.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:00:37.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:00:37.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:00:37.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:00:37.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:37.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:00:37.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:37.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:00:37.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T12:59:04.068459+0000 '>' 2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:37.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:00:38.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:00:38.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:00:38.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:00:38.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:00:38.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:00:38.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:00:38.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:00:38.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:00:32.347912+0000 '>' 2026-02-19T12:59:04.068459+0000 2026-02-19T13:00:38.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:00:38.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4472: corrupt_scrub_erasure: rados list-inconsistent-pg ecpool 2026-02-19T13:00:38.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4474: corrupt_scrub_erasure: jq '. | length' td/osd-scrub-repair/json 2026-02-19T13:00:38.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4474: corrupt_scrub_erasure: test 1 = 1 2026-02-19T13:00:38.978 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4476: corrupt_scrub_erasure: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-19T13:00:38.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4476: corrupt_scrub_erasure: test 3.0 = 3.0 2026-02-19T13:00:38.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4478: corrupt_scrub_erasure: rados list-inconsistent-obj 3.0 2026-02-19T13:00:39.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4479: corrupt_scrub_erasure: cp td/osd-scrub-repair/json /tmp/rrr6.json 2026-02-19T13:00:39.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4481: corrupt_scrub_erasure: jq .epoch td/osd-scrub-repair/json 2026-02-19T13:00:39.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4481: corrupt_scrub_erasure: epoch=101 2026-02-19T13:00:39.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4483: corrupt_scrub_erasure: '[' false = true ']' 2026-02-19T13:00:39.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5148: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-19T13:00:39.021 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:00:39.021 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:00:39.022 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:00:39.022 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:00:39.022 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:00:39.022 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:00:39.022 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:00:39.022 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:00:39.022 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:00:39.022 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:00:39.022 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:00:39.022 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-19T13:00:39.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5148: corrupt_scrub_erasure: jq .inconsistents 2026-02-19T13:00:39.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5148: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:00:39.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5890: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-19T13:00:39.039 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:00:39.039 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:00:39.039 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:00:39.039 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:00:39.039 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:00:39.039 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:00:39.039 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:00:39.039 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:00:39.039 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:00:39.039 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:00:39.039 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:00:39.039 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-19T13:00:39.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5890: corrupt_scrub_erasure: jq .inconsistents 2026-02-19T13:00:39.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5890: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:00:39.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: corrupt_scrub_erasure: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-19T13:00:39.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-19T13:00:39.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5892: corrupt_scrub_erasure: test no = yes 2026-02-19T13:00:39.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5903: corrupt_scrub_erasure: test '' = yes 2026-02-19T13:00:39.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5908: corrupt_scrub_erasure: ceph osd pool rm ecpool ecpool --yes-i-really-really-mean-it 2026-02-19T13:00:39.273 INFO:tasks.workunit.client.0.trial144.stderr:pool 'ecpool' removed 2026-02-19T13:00:39.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T13:00:39.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:00:39.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:00:39.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:00:39.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:00:39.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:00:39.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:00:39.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:00:39.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:00:39.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:00:39.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:00:39.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:00:39.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:00:39.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:00:39.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:00:39.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:00:39.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:00:39.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:00:39.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:00:39.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:00:39.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:00:39.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:00:39.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:00:39.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:00:39.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:00:39.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:00:39.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:00:39.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:00:39.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:00:39.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T13:00:39.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T13:00:39.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T13:00:39.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T13:00:39.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:00:39.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:00:39.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:00:39.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:00:39.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:00:39.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:00:39.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:00:39.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:00:39.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:00:39.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:00:39.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:00:39.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:00:39.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:00:39.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:00:39.440 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:00:39.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:00:39.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:00:39.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:00:39.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:00:39.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:00:39.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:00:39.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:00:39.446 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:00:39.446 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:00:39.446 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:00:39.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:00:39.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:00:39.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:00:39.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T13:00:39.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:00:39.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:00:39.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:00:39.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T13:00:39.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:00:39.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:00:39.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:00:39.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T13:00:39.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_scrub_erasure_overwrites td/osd-scrub-repair 2026-02-19T13:00:39.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5916: TEST_corrupt_scrub_erasure_overwrites: '[' true = true ']' 2026-02-19T13:00:39.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5917: TEST_corrupt_scrub_erasure_overwrites: corrupt_scrub_erasure td/osd-scrub-repair true 2026-02-19T13:00:39.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3736: corrupt_scrub_erasure: local dir=td/osd-scrub-repair 2026-02-19T13:00:39.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3737: corrupt_scrub_erasure: local allow_overwrites=true 2026-02-19T13:00:39.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3738: corrupt_scrub_erasure: local poolname=ecpool 2026-02-19T13:00:39.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3739: corrupt_scrub_erasure: local total_objs=7 2026-02-19T13:00:39.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3741: corrupt_scrub_erasure: run_mon td/osd-scrub-repair a 2026-02-19T13:00:39.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T13:00:39.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:00:39.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:00:39.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:00:39.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T13:00:39.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T13:00:39.473 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:00:39.473 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:00:39.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:00:39.474 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:00:39.474 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:00:39.474 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:00:39.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:00:39.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:00:39.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:00:39.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:00:39.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:00:39.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:00:39.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:00:39.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:00:39.505 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:00:39.505 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:00:39.505 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:00:39.506 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:00:39.506 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:00:39.506 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:00:39.506 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:00:39.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:00:39.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T13:00:39.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:00:39.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:00:39.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:00:39.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:00:39.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:00:39.562 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:00:39.562 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:00:39.563 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:00:39.563 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:00:39.563 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:00:39.563 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:00:39.563 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:00:39.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:00:39.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T13:00:39.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3742: corrupt_scrub_erasure: run_mgr td/osd-scrub-repair x 2026-02-19T13:00:39.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T13:00:39.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:00:39.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:00:39.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:00:39.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T13:00:39.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:00:39.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:00:39.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:00:39.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:00:39.709 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:00:39.709 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:00:39.709 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:00:39.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:00:39.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:00:39.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:00:39.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3743: corrupt_scrub_erasure: seq 0 2 2026-02-19T13:00:39.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3743: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-19T13:00:39.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3744: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 0 2026-02-19T13:00:39.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:00:39.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:00:39.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:00:39.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:00:39.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:00:39.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:00:39.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:00:39.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:00:39.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:00:39.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:00:39.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:00:39.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:00:39.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:00:39.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:00:39.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:00:39.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:00:39.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:00:39.736 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:00:39.736 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:00:39.736 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:00:39.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:00:39.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:00:39.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:00:39.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:00:39.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:00:39.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:00:39.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:00:39.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:00:39.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:00:39.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:00:39.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:00:39.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:00:39.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:00:39.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:00:39.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:00:39.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9466bb56-717c-4025-8eb0-90ced13b4bba 2026-02-19T13:00:39.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 9466bb56-717c-4025-8eb0-90ced13b4bba' 2026-02-19T13:00:39.738 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 9466bb56-717c-4025-8eb0-90ced13b4bba 2026-02-19T13:00:39.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:00:39.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB3CZdpIVGgLBAAJjlKDmDuU/ml8IJkF+r3yA== 2026-02-19T13:00:39.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB3CZdpIVGgLBAAJjlKDmDuU/ml8IJkF+r3yA=="}' 2026-02-19T13:00:39.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9466bb56-717c-4025-8eb0-90ced13b4bba -i td/osd-scrub-repair/0/new.json 2026-02-19T13:00:39.840 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:00:39.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T13:00:39.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB3CZdpIVGgLBAAJjlKDmDuU/ml8IJkF+r3yA== --osd-uuid 9466bb56-717c-4025-8eb0-90ced13b4bba 2026-02-19T13:00:39.865 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:39.864+0000 7f2df4eb0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:39.867 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:39.868+0000 7f2df4eb0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:39.868 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:39.868+0000 7f2df4eb0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:39.868 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:39.868+0000 7f2df4eb0a40 -1 bdev(0x55e33cccac00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:00:39.869 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:39.868+0000 7f2df4eb0a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T13:00:40.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T13:00:40.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:00:40.610 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:00:40.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:00:40.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:00:40.809 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:00:40.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:00:40.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:00:40.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:00:40.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:00:40.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:00:40.824 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:40.824+0000 7f0891cb5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:40.826 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:40.824+0000 7f0891cb5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:40.827 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:40.828+0000 7f0891cb5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:40.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:00:40.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:00:40.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:00:40.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:00:40.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:00:40.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:00:40.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:00:40.974 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:00:40.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:00:40.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:00:41.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:00:41.176 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:41.176+0000 7f0891cb5a40 -1 Falling back to public interface 2026-02-19T13:00:41.744 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:41.744+0000 7f0891cb5a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:00:42.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:00:42.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:00:42.128 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:00:42.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:00:42.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:00:42.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:00:42.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:00:43.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:00:43.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:00:43.288 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:00:43.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:00:43.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:00:43.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:00:43.440 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4214794015,v1:127.0.0.1:6803/4214794015] [v2:127.0.0.1:6804/4214794015,v1:127.0.0.1:6805/4214794015] exists,up 9466bb56-717c-4025-8eb0-90ced13b4bba 2026-02-19T13:00:43.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:00:43.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:00:43.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:00:43.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3743: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-19T13:00:43.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3744: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 1 2026-02-19T13:00:43.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:00:43.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:00:43.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:00:43.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:00:43.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:00:43.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:00:43.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:00:43.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:00:43.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:00:43.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:00:43.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:00:43.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:00:43.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:00:43.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:00:43.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:00:43.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:00:43.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:00:43.445 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:00:43.445 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:00:43.445 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:00:43.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:00:43.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:00:43.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:00:43.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:00:43.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:00:43.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:00:43.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:00:43.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:00:43.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:00:43.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:00:43.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:00:43.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:00:43.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:00:43.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:00:43.447 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:00:43.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7efc0e09-936a-4da0-9723-79c114c06fdf 2026-02-19T13:00:43.447 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 7efc0e09-936a-4da0-9723-79c114c06fdf 2026-02-19T13:00:43.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 7efc0e09-936a-4da0-9723-79c114c06fdf' 2026-02-19T13:00:43.448 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:00:43.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB7CZdpbc9QGxAAzSR+0sATj8jCIT0c2Gd4Ow== 2026-02-19T13:00:43.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB7CZdpbc9QGxAAzSR+0sATj8jCIT0c2Gd4Ow=="}' 2026-02-19T13:00:43.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7efc0e09-936a-4da0-9723-79c114c06fdf -i td/osd-scrub-repair/1/new.json 2026-02-19T13:00:43.616 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:00:43.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T13:00:43.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB7CZdpbc9QGxAAzSR+0sATj8jCIT0c2Gd4Ow== --osd-uuid 7efc0e09-936a-4da0-9723-79c114c06fdf 2026-02-19T13:00:43.644 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:43.644+0000 7f18e9bada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:43.645 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:43.644+0000 7f18e9bada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:43.646 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:43.644+0000 7f18e9bada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:43.647 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:43.648+0000 7f18e9bada40 -1 bdev(0x559ba5c26c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:00:43.647 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:43.648+0000 7f18e9bada40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T13:00:44.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T13:00:44.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:00:44.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:00:44.315 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:00:44.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:00:44.505 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:00:44.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:00:44.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:00:44.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:00:44.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:00:44.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:00:44.530 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:44.528+0000 7f19d4290a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:44.532 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:44.532+0000 7f19d4290a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:44.533 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:44.532+0000 7f19d4290a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:44.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:00:44.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:00:44.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:00:44.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:00:44.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:00:44.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:00:44.669 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:00:44.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:00:44.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:00:44.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:00:44.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:00:44.936 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:44.936+0000 7f19d4290a40 -1 Falling back to public interface 2026-02-19T13:00:45.561 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:45.560+0000 7f19d4290a40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:00:45.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:00:45.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:00:45.833 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:00:45.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:00:45.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:00:45.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:00:45.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:00:46.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:00:46.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:00:46.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:00:46.990 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:00:46.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:00:46.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:00:47.143 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1981054860,v1:127.0.0.1:6811/1981054860] [v2:127.0.0.1:6812/1981054860,v1:127.0.0.1:6813/1981054860] exists,up 7efc0e09-936a-4da0-9723-79c114c06fdf 2026-02-19T13:00:47.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:00:47.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:00:47.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:00:47.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3743: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3744: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 2 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:00:47.145 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:00:47.146 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:00:47.146 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:00:47.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:00:47.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:00:47.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:00:47.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:00:47.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:00:47.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:00:47.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:00:47.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:00:47.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:00:47.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:00:47.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:00:47.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:00:47.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:00:47.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T13:00:47.148 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:00:47.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6f93f708-2939-441c-8ea6-66689f62b2d1 2026-02-19T13:00:47.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 6f93f708-2939-441c-8ea6-66689f62b2d1' 2026-02-19T13:00:47.149 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 6f93f708-2939-441c-8ea6-66689f62b2d1 2026-02-19T13:00:47.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:00:47.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB/CZdpER+rCRAA87lFCfw8+hOB/LXvm70P4A== 2026-02-19T13:00:47.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB/CZdpER+rCRAA87lFCfw8+hOB/LXvm70P4A=="}' 2026-02-19T13:00:47.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6f93f708-2939-441c-8ea6-66689f62b2d1 -i td/osd-scrub-repair/2/new.json 2026-02-19T13:00:47.327 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:00:47.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T13:00:47.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB/CZdpER+rCRAA87lFCfw8+hOB/LXvm70P4A== --osd-uuid 6f93f708-2939-441c-8ea6-66689f62b2d1 2026-02-19T13:00:47.354 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:47.351+0000 7feecc757a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:47.355 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:47.355+0000 7feecc757a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:47.356 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:47.355+0000 7feecc757a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:47.357 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:47.355+0000 7feecc757a40 -1 bdev(0x5597d85bec00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:00:47.357 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:47.355+0000 7feecc757a40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T13:00:48.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T13:00:48.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:00:48.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:00:48.002 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T13:00:48.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:00:48.204 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T13:00:48.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:00:48.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:00:48.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:00:48.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:00:48.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:00:48.229 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:48.227+0000 7f1815952a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:48.230 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:48.231+0000 7f1815952a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:48.231 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:48.231+0000 7f1815952a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:00:48.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:00:48.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:00:48.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:00:48.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:00:48.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:00:48.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:00:48.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:00:48.368 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:00:48.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:00:48.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:00:48.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:00:48.596 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:48.595+0000 7f1815952a40 -1 Falling back to public interface 2026-02-19T13:00:49.208 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:00:49.207+0000 7f1815952a40 -1 osd.2 0 log_to_monitors true 2026-02-19T13:00:49.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:00:49.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:00:49.526 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:00:49.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:00:49.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:00:49.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:00:49.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:00:50.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:00:50.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:00:50.682 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:00:50.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:00:50.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:00:50.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:00:50.837 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/78327750,v1:127.0.0.1:6819/78327750] [v2:127.0.0.1:6820/78327750,v1:127.0.0.1:6821/78327750] exists,up 6f93f708-2939-441c-8ea6-66689f62b2d1 2026-02-19T13:00:50.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:00:50.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:00:50.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:00:50.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3746: corrupt_scrub_erasure: create_rbd_pool 2026-02-19T13:00:50.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T13:00:50.984 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T13:00:50.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T13:00:50.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:00:51.215 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T13:00:51.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:00:52.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T13:00:52.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3747: corrupt_scrub_erasure: create_pool foo 1 2026-02-19T13:00:52.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 1 2026-02-19T13:00:52.747 INFO:tasks.workunit.client.0.trial144.stderr:pool 'foo' created 2026-02-19T13:00:52.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:00:53.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3749: corrupt_scrub_erasure: create_ec_pool ecpool true k=2 m=1 stripe_unit=2K --force 2026-02-19T13:00:53.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: create_ec_pool: local pool_name=ecpool 2026-02-19T13:00:53.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: create_ec_pool: shift 2026-02-19T13:00:53.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: create_ec_pool: local allow_overwrites=true 2026-02-19T13:00:53.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2525: create_ec_pool: shift 2026-02-19T13:00:53.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 stripe_unit=2K --force 2026-02-19T13:00:53.962 INFO:tasks.workunit.client.0.trial144.stderr:load: isa 2026-02-19T13:00:53.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-19T13:00:53.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-19T13:00:54.268 INFO:tasks.workunit.client.0.trial144.stderr:pool 'ecpool' created 2026-02-19T13:00:54.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:00:55.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: create_ec_pool: '[' true = true ']' 2026-02-19T13:00:55.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2532: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-19T13:00:55.484 INFO:tasks.workunit.client.0.trial144.stderr:set pool 3 allow_ec_overwrites to true 2026-02-19T13:00:55.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2535: create_ec_pool: wait_for_clean 2026-02-19T13:00:55.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:00:55.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:00:55.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:00:55.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:00:55.495 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:00:55.495 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:00:55.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:00:55.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:00:55.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:00:55.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:00:55.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:00:55.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:00:55.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:00:55.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:00:55.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:00:55.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:00:55.779 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:00:55.779 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:00:55.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:00:55.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:55.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:00:55.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-19T13:00:55.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-19T13:00:55.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-19T13:00:55.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:55.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:00:55.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-19T13:00:55.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-19T13:00:55.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-38654705667' 2026-02-19T13:00:55.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:55.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:00:55.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542147 2026-02-19T13:00:55.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542147 2026-02-19T13:00:55.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-38654705667 2-60129542147' 2026-02-19T13:00:55.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:55.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-19T13:00:55.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:55.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:00:55.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-19T13:00:55.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:56.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-19T13:00:56.000 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836484 2026-02-19T13:00:56.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-19T13:00:56.001 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:56.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-19T13:00:56.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:00:57.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:00:57.172 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:57.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-19T13:00:57.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:57.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-19T13:00:57.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:57.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:00:57.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705667 2026-02-19T13:00:57.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:57.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705667 2026-02-19T13:00:57.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-19T13:00:57.336 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705667 2026-02-19T13:00:57.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:00:57.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705667 -lt 38654705667 2026-02-19T13:00:57.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:57.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542147 2026-02-19T13:00:57.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:57.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:00:57.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542147 2026-02-19T13:00:57.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:57.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542147 2026-02-19T13:00:57.502 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542147 2026-02-19T13:00:57.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-19T13:00:57.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:00:57.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-19T13:00:57.671 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:00:57.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:00:57.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:00:57.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:00:57.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:00:57.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:00:57.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:00:57.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:00:57.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:00:57.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:00:57.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:00:58.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:00:58.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:00:58.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:00:58.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:00:58.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:00:58.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:00:58.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:00:58.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2536: create_ec_pool: return 0 2026-02-19T13:00:58.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3750: corrupt_scrub_erasure: wait_for_clean 2026-02-19T13:00:58.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:00:58.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:00:58.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:00:58.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:00:58.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:00:58.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:00:58.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:00:58.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:00:58.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:00:58.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:00:58.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:00:58.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:00:58.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:00:58.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:00:58.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:00:58.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:00:58.492 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:00:58.492 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:00:58.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:00:58.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:58.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:00:58.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836486 2026-02-19T13:00:58.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836486 2026-02-19T13:00:58.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836486' 2026-02-19T13:00:58.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:58.568 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:00:58.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705669 2026-02-19T13:00:58.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705669 2026-02-19T13:00:58.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836486 1-38654705669' 2026-02-19T13:00:58.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:00:58.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:00:58.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542148 2026-02-19T13:00:58.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542148 2026-02-19T13:00:58.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836486 1-38654705669 2-60129542148' 2026-02-19T13:00:58.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:00:58.713 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836486 2026-02-19T13:00:58.713 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:00:58.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:00:58.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836486 2026-02-19T13:00:58.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:00:58.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836486 2026-02-19T13:00:58.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-19T13:00:58.717 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836486 2026-02-19T13:00:58.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:00:58.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-19T13:00:58.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:00:59.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:00:59.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:00.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-19T13:01:00.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:01.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:01:01.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:01.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-19T13:01:01.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:01.210 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705669 2026-02-19T13:01:01.210 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:01.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:01:01.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705669 2026-02-19T13:01:01.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:01.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705669 2026-02-19T13:01:01.215 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705669 2026-02-19T13:01:01.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-19T13:01:01.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:01:01.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-19T13:01:01.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:01.376 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542148 2026-02-19T13:01:01.376 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:01.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:01:01.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542148 2026-02-19T13:01:01.379 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:01.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542148 2026-02-19T13:01:01.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-19T13:01:01.381 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542148 2026-02-19T13:01:01.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:01:01.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-19T13:01:01.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:01:01.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:01.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:01.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:01:01.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:01:01.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:01:01.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:01:01.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:01:01.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:01:01.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:01:01.744 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:01:01.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:01:01.903 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:01:01.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:01.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:02.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:01:02.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:01:02.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:01:02.098 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: seq 1 7 2026-02-19T13:01:02.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-19T13:01:02.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3753: corrupt_scrub_erasure: objname=EOBJ1 2026-02-19T13:01:02.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3754: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ1 2026-02-19T13:01:02.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:01:02.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T13:01:02.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ1 2026-02-19T13:01:02.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:01:02.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:01:02.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:01:02.342 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:01:02.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:01:02.542 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:01:02.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:01:02.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:01:02.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:01:02.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: expr 1 % 2 2026-02-19T13:01:02.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: local osd=1 2026-02-19T13:01:02.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3758: corrupt_scrub_erasure: case $i in 2026-02-19T13:01:02.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3761: corrupt_scrub_erasure: local payload=UVWXYZZZ 2026-02-19T13:01:02.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3762: corrupt_scrub_erasure: echo UVWXYZZZ 2026-02-19T13:01:02.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3763: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:01:02.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:01:02.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:01:02.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:01:02.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:01:02.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:01:02.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:01:02.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:01:02.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:01:02.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:01:02.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:01:02.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:01:02.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:01:02.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:01:02.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:01:02.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:01:02.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:01:02.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:01:02.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:01:02.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:01:02.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:01:02.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:01:02.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:01:02.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:01:03.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:01:03.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:01:03.186 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:01:03.186 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:01:03.186 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:01:03.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:03.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:03.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:01:03.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:01:03.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:01:03.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:01:03.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:01:03.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:01:03.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:01:03.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:01:03.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:01:03.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:01:03.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:01:03.188 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:01:03.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:01:03.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:01:03.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:01:03.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:01:03.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:01:03.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:01:03.204 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:03.203+0000 7f939dadaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:03.206 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:03.203+0000 7f939dadaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:03.207 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:03.207+0000 7f939dadaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:03.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:01:03.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:01:03.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:01:03.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:01:03.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:01:03.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:03.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:01:03.343 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:01:03.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:03.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:01:03.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:01:03.588 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:03.587+0000 7f939dadaa40 -1 Falling back to public interface 2026-02-19T13:01:04.198 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:04.195+0000 7f939dadaa40 -1 osd.1 32 log_to_monitors true 2026-02-19T13:01:04.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:01:04.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:04.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:01:04.507 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:04.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:04.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:01:04.667 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 36 up_thru 36 down_at 33 last_clean_interval [9,32) [v2:127.0.0.1:6810/2097969663,v1:127.0.0.1:6811/2097969663] [v2:127.0.0.1:6812/2097969663,v1:127.0.0.1:6813/2097969663] exists,up 7efc0e09-936a-4da0-9723-79c114c06fdf 2026-02-19T13:01:04.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:01:04.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:01:04.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:01:04.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:01:04.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:01:04.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:01:04.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:01:04.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:01:04.668 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:01:04.668 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:01:04.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:01:04.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:01:04.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:01:04.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:01:04.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:01:04.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:01:04.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:01:04.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:01:04.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:01:04.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:01:04.947 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:04.947 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:01:04.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:01:04.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:04.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:01:05.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836488 2026-02-19T13:01:05.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836488 2026-02-19T13:01:05.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488' 2026-02-19T13:01:05.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:05.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:01:05.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822658 2026-02-19T13:01:05.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822658 2026-02-19T13:01:05.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-154618822658' 2026-02-19T13:01:05.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:05.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:01:05.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542150 2026-02-19T13:01:05.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542150 2026-02-19T13:01:05.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-154618822658 2-60129542150' 2026-02-19T13:01:05.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:05.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836488 2026-02-19T13:01:05.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:05.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:01:05.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836488 2026-02-19T13:01:05.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:05.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836488 2026-02-19T13:01:05.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-19T13:01:05.170 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836488 2026-02-19T13:01:05.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:05.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836488 2026-02-19T13:01:05.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:06.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:01:06.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:06.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-19T13:01:06.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:06.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-154618822658 2026-02-19T13:01:06.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:06.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:01:06.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-154618822658 2026-02-19T13:01:06.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:06.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=154618822658 2026-02-19T13:01:06.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 154618822658' 2026-02-19T13:01:06.507 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 154618822658 2026-02-19T13:01:06.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:01:06.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 154618822658 -lt 154618822658 2026-02-19T13:01:06.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:06.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542150 2026-02-19T13:01:06.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:06.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:01:06.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542150 2026-02-19T13:01:06.671 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:06.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542150 2026-02-19T13:01:06.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-19T13:01:06.672 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542150 2026-02-19T13:01:06.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:01:06.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542151 -lt 60129542150 2026-02-19T13:01:06.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:01:06.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:06.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:07.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:01:07.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:01:07.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:01:07.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:01:07.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:01:07.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:01:07.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:01:07.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:01:07.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:01:07.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:01:07.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:07.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:07.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:01:07.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:01:07.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:01:07.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-19T13:01:07.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3753: corrupt_scrub_erasure: objname=EOBJ2 2026-02-19T13:01:07.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3754: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ2 2026-02-19T13:01:07.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:01:07.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T13:01:07.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ2 2026-02-19T13:01:07.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:01:07.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:01:07.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:01:07.577 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:01:07.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:01:07.781 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:01:07.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:01:07.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:01:07.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ2 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:01:07.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: expr 2 % 2 2026-02-19T13:01:07.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: local osd=0 2026-02-19T13:01:07.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3758: corrupt_scrub_erasure: case $i in 2026-02-19T13:01:07.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3768: corrupt_scrub_erasure: dd if=/dev/urandom of=td/osd-scrub-repair/CORRUPT bs=2048 count=1 2026-02-19T13:01:07.817 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:01:07.818 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:01:07.818 INFO:tasks.workunit.client.0.trial144.stderr:2048 bytes (2.0 kB, 2.0 KiB) copied, 9.2341e-05 s, 22.2 MB/s 2026-02-19T13:01:07.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3769: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:01:07.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:01:07.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:01:07.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:01:07.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:01:07.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:01:07.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:01:07.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:01:07.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:01:07.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:01:07.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:01:07.819 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:01:07.819 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:01:07.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:01:07.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:01:07.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:01:08.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:01:08.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:01:08.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:01:08.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:01:08.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:01:08.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:01:08.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:01:08.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:01:08.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:01:08.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:01:08.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:01:08.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:01:08.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:01:08.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:01:08.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:01:08.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:01:08.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:01:08.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:01:08.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:01:08.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:01:08.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:01:08.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:01:08.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:01:08.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:01:08.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:01:08.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:01:08.409 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:01:08.409 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:01:08.409 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:01:08.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:08.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:08.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:01:08.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:01:08.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:01:08.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:01:08.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:01:08.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:01:08.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:01:08.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:01:08.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:01:08.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:01:08.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:01:08.410 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:01:08.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:01:08.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:01:08.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:01:08.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:01:08.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:01:08.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:01:08.424 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:08.423+0000 7f63e5cf4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:08.426 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:08.423+0000 7f63e5cf4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:08.427 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:08.427+0000 7f63e5cf4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:08.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:01:08.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:01:08.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:01:08.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:01:08.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:01:08.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:08.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:01:08.573 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:01:08.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:08.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:01:08.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:01:08.868 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:08.867+0000 7f63e5cf4a40 -1 Falling back to public interface 2026-02-19T13:01:09.463 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:09.459+0000 7f63e5cf4a40 -1 osd.0 39 log_to_monitors true 2026-02-19T13:01:09.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:01:09.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:09.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:01:09.735 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:09.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:09.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:01:09.889 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 43 up_thru 43 down_at 40 last_clean_interval [5,39) [v2:127.0.0.1:6802/2342187574,v1:127.0.0.1:6803/2342187574] [v2:127.0.0.1:6804/2342187574,v1:127.0.0.1:6805/2342187574] exists,up 9466bb56-717c-4025-8eb0-90ced13b4bba 2026-02-19T13:01:09.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:01:09.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:01:09.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:01:09.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:01:09.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:01:09.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:01:09.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:01:09.890 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:01:09.890 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:01:09.890 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:01:09.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:01:09.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:01:09.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:01:10.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:01:10.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:01:10.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:01:10.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:01:10.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:01:10.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:01:10.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:01:10.171 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:10.171 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:01:10.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:01:10.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:10.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:01:10.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593730 2026-02-19T13:01:10.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593730 2026-02-19T13:01:10.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-184683593730' 2026-02-19T13:01:10.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:10.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:01:10.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822660 2026-02-19T13:01:10.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822660 2026-02-19T13:01:10.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-184683593730 1-154618822660' 2026-02-19T13:01:10.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:10.318 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:01:10.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542153 2026-02-19T13:01:10.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542153 2026-02-19T13:01:10.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-184683593730 1-154618822660 2-60129542153' 2026-02-19T13:01:10.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:10.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-184683593730 2026-02-19T13:01:10.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:10.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:01:10.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-184683593730 2026-02-19T13:01:10.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:10.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593730 2026-02-19T13:01:10.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 184683593730' 2026-02-19T13:01:10.396 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 184683593730 2026-02-19T13:01:10.396 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:10.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 184683593730 2026-02-19T13:01:10.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:11.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:01:11.555 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:11.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 184683593730 2026-02-19T13:01:11.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:12.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:01:12.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:12.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593730 -lt 184683593730 2026-02-19T13:01:12.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:12.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-154618822660 2026-02-19T13:01:12.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:12.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:01:12.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-154618822660 2026-02-19T13:01:12.881 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:12.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=154618822660 2026-02-19T13:01:12.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 154618822660' 2026-02-19T13:01:12.883 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 154618822660 2026-02-19T13:01:12.883 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:01:13.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 154618822660 -lt 154618822660 2026-02-19T13:01:13.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:13.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542153 2026-02-19T13:01:13.054 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:13.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:01:13.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542153 2026-02-19T13:01:13.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:13.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542153 2026-02-19T13:01:13.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-19T13:01:13.058 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542153 2026-02-19T13:01:13.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:01:13.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-19T13:01:13.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:01:13.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:13.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:13.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:01:13.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:01:13.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:01:13.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:01:13.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:01:13.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:01:13.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:01:13.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:01:13.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:01:13.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:01:13.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:13.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:13.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:01:13.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:01:13.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:01:13.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-19T13:01:13.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3753: corrupt_scrub_erasure: objname=EOBJ3 2026-02-19T13:01:13.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3754: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ3 2026-02-19T13:01:13.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:01:13.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T13:01:13.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ3 2026-02-19T13:01:13.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:01:13.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:01:13.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:01:13.976 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:01:13.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:01:14.180 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:01:14.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:01:14.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:01:14.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ3 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:01:14.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: expr 3 % 2 2026-02-19T13:01:14.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: local osd=1 2026-02-19T13:01:14.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3758: corrupt_scrub_erasure: case $i in 2026-02-19T13:01:14.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3774: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ3 remove 2026-02-19T13:01:14.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:01:14.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:01:14.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:01:14.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:01:14.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ3 remove 2026-02-19T13:01:14.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:01:14.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:01:14.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:01:14.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:01:14.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:01:14.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:01:14.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:01:14.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:01:14.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:01:14.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:01:14.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:01:14.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ3 remove 2026-02-19T13:01:14.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:01:14.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:01:14.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:01:14.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:01:14.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:01:14.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ3 remove 2026-02-19T13:01:14.703 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#3:b197b25d:::EOBJ3:head# 2026-02-19T13:01:14.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:01:14.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:01:14.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:01:14.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:01:14.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:01:14.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:01:14.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:01:14.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:01:14.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:01:14.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:01:14.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:01:14.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:01:14.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:01:14.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:01:14.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:01:14.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:01:14.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:01:14.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:01:14.818 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:01:14.818 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:01:14.818 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:01:14.820 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:14.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:14.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:01:14.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:01:14.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:01:14.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:01:14.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:01:14.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:01:14.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:01:14.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:01:14.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:01:14.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:01:14.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:01:14.821 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:01:14.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:01:14.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:01:14.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:01:14.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:01:14.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:01:14.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:01:14.842 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:14.839+0000 7f291c1c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:14.844 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:14.843+0000 7f291c1c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:14.845 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:14.843+0000 7f291c1c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:14.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:01:14.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:01:14.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:01:14.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:01:14.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:01:14.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:14.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:01:14.983 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:01:14.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:14.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:01:15.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:01:15.236 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:15.235+0000 7f291c1c7a40 -1 Falling back to public interface 2026-02-19T13:01:15.864 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:15.863+0000 7f291c1c7a40 -1 osd.1 47 log_to_monitors true 2026-02-19T13:01:16.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:01:16.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:16.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:01:16.144 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:16.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:16.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:01:16.299 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 51 up_thru 51 down_at 48 last_clean_interval [36,47) [v2:127.0.0.1:6810/3116897982,v1:127.0.0.1:6811/3116897982] [v2:127.0.0.1:6812/3116897982,v1:127.0.0.1:6813/3116897982] exists,up 7efc0e09-936a-4da0-9723-79c114c06fdf 2026-02-19T13:01:16.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:01:16.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:01:16.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:01:16.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:01:16.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:01:16.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:01:16.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:01:16.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:01:16.300 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:01:16.301 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:01:16.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:01:16.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:01:16.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:01:16.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:01:16.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:01:16.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:01:16.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:01:16.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:01:16.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:01:16.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:01:16.577 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:16.577 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:01:16.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:01:16.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:16.577 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:01:16.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593732 2026-02-19T13:01:16.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593732 2026-02-19T13:01:16.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-184683593732' 2026-02-19T13:01:16.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:16.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:01:16.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332098 2026-02-19T13:01:16.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332098 2026-02-19T13:01:16.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-184683593732 1-219043332098' 2026-02-19T13:01:16.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:16.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:01:16.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542155 2026-02-19T13:01:16.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542155 2026-02-19T13:01:16.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-184683593732 1-219043332098 2-60129542155' 2026-02-19T13:01:16.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:16.800 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-184683593732 2026-02-19T13:01:16.800 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:16.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:01:16.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-184683593732 2026-02-19T13:01:16.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:16.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593732 2026-02-19T13:01:16.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 184683593732' 2026-02-19T13:01:16.804 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 184683593732 2026-02-19T13:01:16.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:16.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593731 -lt 184683593732 2026-02-19T13:01:16.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:17.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:01:17.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:18.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593731 -lt 184683593732 2026-02-19T13:01:18.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:19.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:01:19.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:19.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593732 -lt 184683593732 2026-02-19T13:01:19.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:19.289 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-219043332098 2026-02-19T13:01:19.289 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:19.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:01:19.291 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-219043332098 2026-02-19T13:01:19.291 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:19.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=219043332098 2026-02-19T13:01:19.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 219043332098' 2026-02-19T13:01:19.293 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 219043332098 2026-02-19T13:01:19.293 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:01:19.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 219043332098 -lt 219043332098 2026-02-19T13:01:19.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:19.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542155 2026-02-19T13:01:19.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:19.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:01:19.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542155 2026-02-19T13:01:19.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:19.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542155 2026-02-19T13:01:19.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542155' 2026-02-19T13:01:19.459 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542155 2026-02-19T13:01:19.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:01:19.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542155 -lt 60129542155 2026-02-19T13:01:19.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:01:19.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:19.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:19.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:01:19.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:01:19.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:01:19.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:01:19.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:01:19.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:01:19.806 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:01:19.806 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:01:19.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:01:19.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:01:19.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:19.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:20.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:01:20.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:01:20.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:01:20.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-19T13:01:20.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3753: corrupt_scrub_erasure: objname=EOBJ4 2026-02-19T13:01:20.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3754: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ4 2026-02-19T13:01:20.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:01:20.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T13:01:20.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ4 2026-02-19T13:01:20.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:01:20.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:01:20.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:01:20.390 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:01:20.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:01:20.593 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:01:20.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:01:20.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:01:20.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ4 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:01:20.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: expr 4 % 2 2026-02-19T13:01:20.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: local osd=0 2026-02-19T13:01:20.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3758: corrupt_scrub_erasure: case $i in 2026-02-19T13:01:20.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3778: corrupt_scrub_erasure: rados --pool ecpool setxattr EOBJ4 key1-EOBJ4 val1-EOBJ4 2026-02-19T13:01:20.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3779: corrupt_scrub_erasure: rados --pool ecpool setxattr EOBJ4 key2-EOBJ4 val2-EOBJ4 2026-02-19T13:01:20.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3782: corrupt_scrub_erasure: echo -n bad-val 2026-02-19T13:01:20.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3783: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-19T13:01:20.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:01:20.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:01:20.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:01:20.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:01:20.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-19T13:01:20.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:01:20.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:01:20.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:01:20.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:01:20.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:01:20.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:01:20.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:01:20.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:01:20.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:01:20.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:01:20.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:01:20.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-19T13:01:20.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:01:20.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:01:20.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:01:20.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:01:20.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:01:20.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-19T13:01:21.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:01:21.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:01:21.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:01:21.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:01:21.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:01:21.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:01:21.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:01:21.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:01:21.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:01:21.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:01:21.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:01:21.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:01:21.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:01:21.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:01:21.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:01:21.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:01:21.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:01:21.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:01:21.250 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:01:21.250 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:01:21.250 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:01:21.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:21.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:21.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:01:21.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:01:21.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:01:21.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:01:21.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:01:21.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:01:21.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:01:21.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:01:21.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:01:21.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:01:21.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:01:21.254 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:01:21.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:01:21.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:01:21.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:01:21.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:01:21.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:01:21.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:01:21.268 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:21.267+0000 7f56b8255a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:21.270 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:21.267+0000 7f56b8255a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:21.271 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:21.271+0000 7f56b8255a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:21.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:01:21.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:01:21.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:01:21.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:01:21.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:01:21.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:21.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:01:21.411 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:01:21.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:21.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:01:21.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:01:21.656 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:21.655+0000 7f56b8255a40 -1 Falling back to public interface 2026-02-19T13:01:22.232 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:22.231+0000 7f56b8255a40 -1 osd.0 54 log_to_monitors true 2026-02-19T13:01:22.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:01:22.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:22.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:01:22.571 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:22.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:22.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:01:22.735 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 58 up_thru 58 down_at 55 last_clean_interval [43,54) [v2:127.0.0.1:6802/2291803197,v1:127.0.0.1:6803/2291803197] [v2:127.0.0.1:6804/2291803197,v1:127.0.0.1:6805/2291803197] exists,up 9466bb56-717c-4025-8eb0-90ced13b4bba 2026-02-19T13:01:22.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:01:22.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:01:22.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:01:22.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:01:22.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:01:22.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:01:22.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:01:22.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:01:22.737 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:01:22.737 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:01:22.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:01:22.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:01:22.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:01:22.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:01:22.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:01:22.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:01:22.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:01:22.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:01:22.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:01:23.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:01:23.008 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:23.008 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:01:23.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:01:23.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:23.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:01:23.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=249108103170 2026-02-19T13:01:23.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 249108103170 2026-02-19T13:01:23.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103170' 2026-02-19T13:01:23.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:23.080 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:01:23.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332100 2026-02-19T13:01:23.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332100 2026-02-19T13:01:23.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103170 1-219043332100' 2026-02-19T13:01:23.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:23.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:01:23.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542157 2026-02-19T13:01:23.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542157 2026-02-19T13:01:23.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103170 1-219043332100 2-60129542157' 2026-02-19T13:01:23.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:23.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-249108103170 2026-02-19T13:01:23.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:23.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:01:23.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-249108103170 2026-02-19T13:01:23.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:23.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=249108103170 2026-02-19T13:01:23.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 249108103170' 2026-02-19T13:01:23.230 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 249108103170 2026-02-19T13:01:23.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:23.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593733 -lt 249108103170 2026-02-19T13:01:23.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:24.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:01:24.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:24.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103170 -lt 249108103170 2026-02-19T13:01:24.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:24.550 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-219043332100 2026-02-19T13:01:24.550 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:24.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:01:24.552 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-219043332100 2026-02-19T13:01:24.552 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:24.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=219043332100 2026-02-19T13:01:24.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 219043332100' 2026-02-19T13:01:24.554 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 219043332100 2026-02-19T13:01:24.554 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:01:24.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 219043332100 -lt 219043332100 2026-02-19T13:01:24.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:24.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542157 2026-02-19T13:01:24.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:24.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:01:24.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542157 2026-02-19T13:01:24.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:24.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542157 2026-02-19T13:01:24.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542157' 2026-02-19T13:01:24.726 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542157 2026-02-19T13:01:24.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:01:24.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542157 -lt 60129542157 2026-02-19T13:01:24.893 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:01:24.893 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:24.893 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:25.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:01:25.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:01:25.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:01:25.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:01:25.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:01:25.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:01:25.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:01:25.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:01:25.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:01:25.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:01:25.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:25.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:25.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:01:25.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:01:25.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:01:25.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3784: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-19T13:01:25.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:01:25.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:01:25.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T13:01:25.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:01:25.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-19T13:01:25.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:01:25.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:01:25.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T13:01:25.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:01:25.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-19T13:01:25.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:01:25.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:01:25.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:01:25.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:01:25.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:01:25.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:01:25.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-19T13:01:25.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:01:25.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:01:25.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T13:01:25.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:01:25.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:01:25.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-19T13:01:26.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:01:26.029 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:01:26.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:01:26.030 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:01:26.030 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:01:26.030 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:01:26.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:26.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:26.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:01:26.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:01:26.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:01:26.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:01:26.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:01:26.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:01:26.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:01:26.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:01:26.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:01:26.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T13:01:26.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:01:26.032 INFO:tasks.workunit.client.0.trial144.stderr:start osd.2 2026-02-19T13:01:26.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:01:26.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-19T13:01:26.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:01:26.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:01:26.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:01:26.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:01:26.056 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:26.055+0000 7ff06240fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:26.057 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:26.055+0000 7ff06240fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:26.058 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:26.059+0000 7ff06240fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:26.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:01:26.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:01:26.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:01:26.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:01:26.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:01:26.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:26.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:01:26.192 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:01:26.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:26.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:01:26.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:01:26.504 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:26.503+0000 7ff06240fa40 -1 Falling back to public interface 2026-02-19T13:01:27.091 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:27.091+0000 7ff06240fa40 -1 osd.2 60 log_to_monitors true 2026-02-19T13:01:27.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:01:27.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:27.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:01:27.352 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:27.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:27.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:01:27.508 INFO:tasks.workunit.client.0.trial144.stderr:osd.2 up in weight 1 up_from 63 up_thru 63 down_at 60 last_clean_interval [14,60) [v2:127.0.0.1:6818/2351068422,v1:127.0.0.1:6819/2351068422] [v2:127.0.0.1:6820/2351068422,v1:127.0.0.1:6821/2351068422] exists,up 6f93f708-2939-441c-8ea6-66689f62b2d1 2026-02-19T13:01:27.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:01:27.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:01:27.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:01:27.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:01:27.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:01:27.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:01:27.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:01:27.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:01:27.510 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:01:27.510 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:01:27.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:01:27.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:01:27.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:01:27.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:01:27.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:01:27.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:01:27.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:01:27.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:01:27.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:01:27.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:01:27.781 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:27.781 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:01:27.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:01:27.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:27.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:01:27.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=249108103172 2026-02-19T13:01:27.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 249108103172 2026-02-19T13:01:27.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103172' 2026-02-19T13:01:27.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:27.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:01:27.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332102 2026-02-19T13:01:27.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332102 2026-02-19T13:01:27.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103172 1-219043332102' 2026-02-19T13:01:27.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:27.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:01:28.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=270582939650 2026-02-19T13:01:28.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 270582939650 2026-02-19T13:01:28.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103172 1-219043332102 2-270582939650' 2026-02-19T13:01:28.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:28.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-249108103172 2026-02-19T13:01:28.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:28.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:01:28.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-249108103172 2026-02-19T13:01:28.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:28.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=249108103172 2026-02-19T13:01:28.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 249108103172' 2026-02-19T13:01:28.006 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 249108103172 2026-02-19T13:01:28.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:28.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103170 -lt 249108103172 2026-02-19T13:01:28.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:29.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:01:29.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:29.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103172 -lt 249108103172 2026-02-19T13:01:29.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:29.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-219043332102 2026-02-19T13:01:29.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:29.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:01:29.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-219043332102 2026-02-19T13:01:29.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:29.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=219043332102 2026-02-19T13:01:29.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 219043332102' 2026-02-19T13:01:29.338 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 219043332102 2026-02-19T13:01:29.338 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:01:29.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 219043332102 -lt 219043332102 2026-02-19T13:01:29.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:29.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-270582939650 2026-02-19T13:01:29.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:29.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:01:29.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-270582939650 2026-02-19T13:01:29.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:29.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=270582939650 2026-02-19T13:01:29.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 270582939650' 2026-02-19T13:01:29.510 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 270582939650 2026-02-19T13:01:29.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:01:29.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 270582939650 -lt 270582939650 2026-02-19T13:01:29.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:01:29.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:29.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:29.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:01:29.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:01:29.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:01:29.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:01:29.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:01:29.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:01:29.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:01:29.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:01:30.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:01:30.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:01:30.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:30.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:30.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:01:30.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:01:30.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:01:30.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3785: corrupt_scrub_erasure: echo -n val3-EOBJ4 2026-02-19T13:01:30.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3786: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-19T13:01:30.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:01:30.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:01:30.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T13:01:30.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:01:30.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-19T13:01:30.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:01:30.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:01:30.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T13:01:30.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:01:30.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-19T13:01:30.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:01:30.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:01:30.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:01:30.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:01:30.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:01:30.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:01:30.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-19T13:01:30.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:01:30.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:01:30.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T13:01:30.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:01:30.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:01:30.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-19T13:01:30.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-19T13:01:30.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:01:30.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:01:30.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:01:30.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:01:30.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:01:30.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:01:30.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:01:30.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:01:30.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:01:30.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T13:01:30.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T13:01:30.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:01:30.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:01:30.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:01:30.786 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:01:30.786 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:01:30.786 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:01:30.786 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:01:30.787 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:01:30.787 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:01:30.787 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:30.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:30.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:01:30.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:01:30.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:01:30.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:01:30.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:01:30.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:01:30.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:01:30.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:01:30.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:01:30.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T13:01:30.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:01:30.790 INFO:tasks.workunit.client.0.trial144.stderr:start osd.2 2026-02-19T13:01:30.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:01:30.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-19T13:01:30.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:01:30.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:01:30.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:01:30.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:01:30.801 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:30.799+0000 7f5973380a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:30.802 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:30.799+0000 7f5973380a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:30.803 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:30.803+0000 7f5973380a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:30.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:01:30.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:01:30.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:01:30.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:01:30.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:01:30.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:30.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:01:30.952 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:01:30.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:30.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:01:31.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:01:31.144 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:31.143+0000 7f5973380a40 -1 Falling back to public interface 2026-02-19T13:01:31.751 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:31.747+0000 7f5973380a40 -1 osd.2 64 log_to_monitors true 2026-02-19T13:01:32.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:01:32.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:32.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:01:32.107 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:32.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:32.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:01:32.264 INFO:tasks.workunit.client.0.trial144.stderr:osd.2 up in weight 1 up_from 69 up_thru 69 down_at 66 last_clean_interval [63,64) [v2:127.0.0.1:6818/1305564391,v1:127.0.0.1:6819/1305564391] [v2:127.0.0.1:6820/1305564391,v1:127.0.0.1:6821/1305564391] exists,up 6f93f708-2939-441c-8ea6-66689f62b2d1 2026-02-19T13:01:32.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:01:32.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:01:32.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:01:32.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:01:32.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:01:32.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:01:32.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:01:32.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:01:32.266 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:01:32.266 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:01:32.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:01:32.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:01:32.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:01:32.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:01:32.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:01:32.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:01:32.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:01:32.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:01:32.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:01:32.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:01:32.540 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:32.541 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:01:32.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:01:32.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:32.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:01:32.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=249108103174 2026-02-19T13:01:32.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 249108103174 2026-02-19T13:01:32.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103174' 2026-02-19T13:01:32.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:32.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:01:32.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332104 2026-02-19T13:01:32.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332104 2026-02-19T13:01:32.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103174 1-219043332104' 2026-02-19T13:01:32.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:32.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:01:32.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743426 2026-02-19T13:01:32.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743426 2026-02-19T13:01:32.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103174 1-219043332104 2-296352743426' 2026-02-19T13:01:32.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:32.759 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-249108103174 2026-02-19T13:01:32.759 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:32.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:01:32.760 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-249108103174 2026-02-19T13:01:32.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:32.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=249108103174 2026-02-19T13:01:32.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 249108103174' 2026-02-19T13:01:32.763 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 249108103174 2026-02-19T13:01:32.763 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:32.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103172 -lt 249108103174 2026-02-19T13:01:32.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:33.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:01:33.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:34.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103172 -lt 249108103174 2026-02-19T13:01:34.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:35.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:01:35.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:35.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103174 -lt 249108103174 2026-02-19T13:01:35.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:35.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-219043332104 2026-02-19T13:01:35.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:35.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:01:35.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-219043332104 2026-02-19T13:01:35.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:35.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=219043332104 2026-02-19T13:01:35.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 219043332104' 2026-02-19T13:01:35.248 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 219043332104 2026-02-19T13:01:35.249 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:01:35.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 219043332104 -lt 219043332104 2026-02-19T13:01:35.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:35.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743426 2026-02-19T13:01:35.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:35.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:01:35.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743426 2026-02-19T13:01:35.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:35.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743426 2026-02-19T13:01:35.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743426' 2026-02-19T13:01:35.424 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 296352743426 2026-02-19T13:01:35.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:01:35.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743426 -lt 296352743426 2026-02-19T13:01:35.583 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:01:35.584 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:35.584 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:35.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:01:35.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:01:35.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:01:35.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:01:35.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:01:35.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:01:35.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:01:35.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:01:35.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:01:35.939 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:01:35.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:35.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:36.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:01:36.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:01:36.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:01:36.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3787: corrupt_scrub_erasure: rm td/osd-scrub-repair/bad-val td/osd-scrub-repair/newval 2026-02-19T13:01:36.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-19T13:01:36.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3753: corrupt_scrub_erasure: objname=EOBJ5 2026-02-19T13:01:36.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3754: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ5 2026-02-19T13:01:36.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:01:36.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T13:01:36.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ5 2026-02-19T13:01:36.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:01:36.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:01:36.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:01:36.378 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:01:36.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:01:36.580 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:01:36.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:01:36.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:01:36.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ5 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:01:36.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: expr 5 % 2 2026-02-19T13:01:36.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: local osd=1 2026-02-19T13:01:36.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3758: corrupt_scrub_erasure: case $i in 2026-02-19T13:01:36.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3792: corrupt_scrub_erasure: dd if=/dev/urandom of=td/osd-scrub-repair/CORRUPT bs=2048 count=2 2026-02-19T13:01:36.620 INFO:tasks.workunit.client.0.trial144.stderr:2+0 records in 2026-02-19T13:01:36.620 INFO:tasks.workunit.client.0.trial144.stderr:2+0 records out 2026-02-19T13:01:36.620 INFO:tasks.workunit.client.0.trial144.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 9.6399e-05 s, 42.5 MB/s 2026-02-19T13:01:36.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3793: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:01:36.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:01:36.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:01:36.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:01:36.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:01:36.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:01:36.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:01:36.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:01:36.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:01:36.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:01:36.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:01:36.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:01:36.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:01:36.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:01:36.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:01:36.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:01:36.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:01:36.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:01:36.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:01:36.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:01:36.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:01:36.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:01:36.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:01:36.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:01:37.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:01:37.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:01:37.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:01:37.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:01:37.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:01:37.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:01:37.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:01:37.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:01:37.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:01:37.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:01:37.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:01:37.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:01:37.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:01:37.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:01:37.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:01:37.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:01:37.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:01:37.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:01:37.219 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:01:37.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:01:37.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:01:37.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:37.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:37.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:01:37.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:01:37.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:01:37.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:01:37.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:01:37.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:01:37.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:01:37.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:01:37.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:01:37.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:01:37.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:01:37.221 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:01:37.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:01:37.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:01:37.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:01:37.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:01:37.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:01:37.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:01:37.235 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:37.234+0000 7fdb5ad1ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:37.236 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:37.234+0000 7fdb5ad1ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:37.237 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:37.234+0000 7fdb5ad1ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:37.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:01:37.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:01:37.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:01:37.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:01:37.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:01:37.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:37.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:01:37.383 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:01:37.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:37.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:01:37.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:01:37.616 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:37.614+0000 7fdb5ad1ca40 -1 Falling back to public interface 2026-02-19T13:01:38.211 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:38.210+0000 7fdb5ad1ca40 -1 osd.1 73 log_to_monitors true 2026-02-19T13:01:38.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:01:38.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:38.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:01:38.541 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:38.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:38.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:01:38.693 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 77 up_thru 77 down_at 74 last_clean_interval [51,73) [v2:127.0.0.1:6810/1394601611,v1:127.0.0.1:6811/1394601611] [v2:127.0.0.1:6812/1394601611,v1:127.0.0.1:6813/1394601611] exists,up 7efc0e09-936a-4da0-9723-79c114c06fdf 2026-02-19T13:01:38.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:01:38.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:01:38.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:01:38.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:01:38.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:01:38.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:01:38.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:01:38.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:01:38.695 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:01:38.695 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:01:38.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:01:38.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:01:38.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:01:38.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:01:38.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:01:38.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:01:38.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:01:38.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:01:38.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:01:38.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:01:38.968 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:38.968 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:01:38.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:01:38.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:38.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:01:39.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=249108103176 2026-02-19T13:01:39.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 249108103176 2026-02-19T13:01:39.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103176' 2026-02-19T13:01:39.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:39.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:01:39.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=330712481794 2026-02-19T13:01:39.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 330712481794 2026-02-19T13:01:39.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103176 1-330712481794' 2026-02-19T13:01:39.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:39.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:01:39.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743428 2026-02-19T13:01:39.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743428 2026-02-19T13:01:39.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-249108103176 1-330712481794 2-296352743428' 2026-02-19T13:01:39.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:39.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-249108103176 2026-02-19T13:01:39.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:39.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:01:39.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-249108103176 2026-02-19T13:01:39.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:39.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=249108103176 2026-02-19T13:01:39.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 249108103176' 2026-02-19T13:01:39.191 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 249108103176 2026-02-19T13:01:39.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:39.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103175 -lt 249108103176 2026-02-19T13:01:39.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:40.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:01:40.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:40.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103176 -lt 249108103176 2026-02-19T13:01:40.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:40.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-330712481794 2026-02-19T13:01:40.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:40.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:01:40.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-330712481794 2026-02-19T13:01:40.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:40.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=330712481794 2026-02-19T13:01:40.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 330712481794' 2026-02-19T13:01:40.511 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 330712481794 2026-02-19T13:01:40.511 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:01:40.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 330712481794 -lt 330712481794 2026-02-19T13:01:40.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:40.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743428 2026-02-19T13:01:40.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:40.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:01:40.677 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743428 2026-02-19T13:01:40.677 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:40.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743428 2026-02-19T13:01:40.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743428' 2026-02-19T13:01:40.679 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 296352743428 2026-02-19T13:01:40.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:01:40.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743428 -lt 296352743428 2026-02-19T13:01:40.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:01:40.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:40.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:41.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:01:41.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:01:41.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:01:41.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:01:41.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:01:41.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:01:41.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:01:41.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:01:41.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:01:41.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:01:41.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:41.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:41.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:01:41.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:01:41.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:01:41.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-19T13:01:41.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3753: corrupt_scrub_erasure: objname=EOBJ6 2026-02-19T13:01:41.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3754: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ6 2026-02-19T13:01:41.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:01:41.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T13:01:41.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ6 2026-02-19T13:01:41.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:01:41.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:01:41.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:01:41.593 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:01:41.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:01:41.796 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:01:41.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:01:41.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:01:41.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ6 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:01:41.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: expr 6 % 2 2026-02-19T13:01:41.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: local osd=0 2026-02-19T13:01:41.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3758: corrupt_scrub_erasure: case $i in 2026-02-19T13:01:41.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3797: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-19T13:01:41.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:01:41.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:01:41.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:01:41.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:01:41.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-19T13:01:41.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:01:41.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:01:41.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:01:41.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:01:41.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:01:41.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:01:41.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:01:41.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:01:41.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:01:41.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:01:42.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:01:42.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-19T13:01:42.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:01:42.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:01:42.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:01:42.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:01:42.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:01:42.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ6 rm-attr hinfo_key 2026-02-19T13:01:42.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:01:42.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:01:42.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:01:42.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:01:42.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:01:42.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:01:42.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:01:42.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:01:42.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:01:42.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:01:42.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:01:42.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:01:42.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:01:42.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:01:42.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:01:42.418 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:01:42.418 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:01:42.418 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:01:42.418 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:01:42.418 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:01:42.418 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:01:42.420 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:42.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:42.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:01:42.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:01:42.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:01:42.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:01:42.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:01:42.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:01:42.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:01:42.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:01:42.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:01:42.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:01:42.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:01:42.421 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:01:42.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:01:42.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:01:42.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:01:42.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:01:42.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:01:42.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:01:42.438 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:42.434+0000 7f5ef93dca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:42.439 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:42.438+0000 7f5ef93dca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:42.440 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:42.438+0000 7f5ef93dca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:42.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:01:42.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:01:42.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:01:42.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:01:42.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:01:42.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:42.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:01:42.581 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:01:42.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:42.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:01:42.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:01:42.868 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:42.866+0000 7f5ef93dca40 -1 Falling back to public interface 2026-02-19T13:01:43.438 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:43.434+0000 7f5ef93dca40 -1 osd.0 80 log_to_monitors true 2026-02-19T13:01:43.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:01:43.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:43.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:01:43.739 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:43.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:43.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:01:43.894 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 84 up_thru 84 down_at 81 last_clean_interval [58,80) [v2:127.0.0.1:6802/1780725091,v1:127.0.0.1:6803/1780725091] [v2:127.0.0.1:6804/1780725091,v1:127.0.0.1:6805/1780725091] exists,up 9466bb56-717c-4025-8eb0-90ced13b4bba 2026-02-19T13:01:43.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:01:43.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:01:43.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:01:43.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:01:43.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:01:43.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:01:43.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:01:43.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:01:43.895 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:01:43.895 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:01:43.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:01:43.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:01:43.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:01:44.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:01:44.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:01:44.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:01:44.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:01:44.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:01:44.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:01:44.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:01:44.175 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:44.175 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:01:44.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:01:44.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:44.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:01:44.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=360777252866 2026-02-19T13:01:44.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 360777252866 2026-02-19T13:01:44.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-360777252866' 2026-02-19T13:01:44.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:44.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:01:44.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=330712481796 2026-02-19T13:01:44.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 330712481796 2026-02-19T13:01:44.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-360777252866 1-330712481796' 2026-02-19T13:01:44.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:44.318 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:01:44.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743430 2026-02-19T13:01:44.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743430 2026-02-19T13:01:44.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-360777252866 1-330712481796 2-296352743430' 2026-02-19T13:01:44.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:44.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-360777252866 2026-02-19T13:01:44.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:44.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:01:44.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-360777252866 2026-02-19T13:01:44.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:44.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=360777252866 2026-02-19T13:01:44.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 360777252866' 2026-02-19T13:01:44.393 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 360777252866 2026-02-19T13:01:44.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:44.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 360777252866 2026-02-19T13:01:44.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:45.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:01:45.566 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:45.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 360777252866 2026-02-19T13:01:45.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:46.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:01:46.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:46.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 360777252866 -lt 360777252866 2026-02-19T13:01:46.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:46.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-330712481796 2026-02-19T13:01:46.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:46.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:01:46.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-330712481796 2026-02-19T13:01:46.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:46.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=330712481796 2026-02-19T13:01:46.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 330712481796' 2026-02-19T13:01:46.897 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 330712481796 2026-02-19T13:01:46.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:01:47.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 330712481796 -lt 330712481796 2026-02-19T13:01:47.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:47.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743430 2026-02-19T13:01:47.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:47.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:01:47.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743430 2026-02-19T13:01:47.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:47.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743430 2026-02-19T13:01:47.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743430' 2026-02-19T13:01:47.069 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 296352743430 2026-02-19T13:01:47.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:01:47.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743430 -lt 296352743430 2026-02-19T13:01:47.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:01:47.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:47.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:47.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:01:47.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:01:47.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:01:47.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:01:47.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:01:47.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:01:47.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:01:47.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:01:47.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:01:47.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:01:47.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:47.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:47.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3798: corrupt_scrub_erasure: echo -n bad-val 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3799: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:01:47.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:01:48.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:01:48.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-19T13:01:48.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:01:48.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:01:48.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:01:48.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:01:48.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:01:48.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-19T13:01:48.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:01:48.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:01:48.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:01:48.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:01:48.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:01:48.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:01:48.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:01:48.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:01:48.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:01:48.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:01:48.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:01:48.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:01:48.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:01:48.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:01:48.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:01:48.370 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:01:48.370 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:01:48.370 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:01:48.370 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:01:48.371 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:01:48.371 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:01:48.371 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:48.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:48.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:01:48.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:01:48.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:01:48.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:01:48.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:01:48.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:01:48.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:01:48.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:01:48.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:01:48.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:01:48.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:01:48.374 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:01:48.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:01:48.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:01:48.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:01:48.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:01:48.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:01:48.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:01:48.386 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:48.382+0000 7f45963eda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:48.387 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:48.386+0000 7f45963eda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:48.388 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:48.386+0000 7f45963eda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:48.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:01:48.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:01:48.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:01:48.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:01:48.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:01:48.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:48.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:01:48.531 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:01:48.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:48.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:01:48.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:01:48.744 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:48.742+0000 7f45963eda40 -1 Falling back to public interface 2026-02-19T13:01:49.411 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:49.410+0000 7f45963eda40 -1 osd.1 85 log_to_monitors true 2026-02-19T13:01:49.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:01:49.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:49.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:01:49.691 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:49.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:49.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:01:49.849 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 89 up_thru 89 down_at 86 last_clean_interval [77,85) [v2:127.0.0.1:6810/728449888,v1:127.0.0.1:6811/728449888] [v2:127.0.0.1:6812/728449888,v1:127.0.0.1:6813/728449888] exists,up 7efc0e09-936a-4da0-9723-79c114c06fdf 2026-02-19T13:01:49.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:01:49.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:01:49.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:01:49.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:01:49.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:01:49.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:01:49.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:01:49.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:01:49.851 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:01:49.851 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:01:49.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:01:49.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:01:49.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:01:49.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:01:49.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:01:49.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:01:49.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:01:49.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:01:49.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:01:50.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:01:50.124 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:50.124 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:01:50.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:01:50.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:50.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:01:50.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=360777252868 2026-02-19T13:01:50.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 360777252868 2026-02-19T13:01:50.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-360777252868' 2026-02-19T13:01:50.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:50.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:01:50.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=382252089346 2026-02-19T13:01:50.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 382252089346 2026-02-19T13:01:50.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-360777252868 1-382252089346' 2026-02-19T13:01:50.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:50.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:01:50.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743432 2026-02-19T13:01:50.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743432 2026-02-19T13:01:50.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-360777252868 1-382252089346 2-296352743432' 2026-02-19T13:01:50.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:50.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-360777252868 2026-02-19T13:01:50.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:50.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:01:50.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-360777252868 2026-02-19T13:01:50.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:50.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=360777252868 2026-02-19T13:01:50.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 360777252868' 2026-02-19T13:01:50.350 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 360777252868 2026-02-19T13:01:50.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:50.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 360777252868 -lt 360777252868 2026-02-19T13:01:50.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:50.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-382252089346 2026-02-19T13:01:50.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:50.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:01:50.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-382252089346 2026-02-19T13:01:50.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:50.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=382252089346 2026-02-19T13:01:50.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 382252089346' 2026-02-19T13:01:50.523 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 382252089346 2026-02-19T13:01:50.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:01:50.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 382252089346 2026-02-19T13:01:50.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:51.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:01:51.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:01:51.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 382252089346 2026-02-19T13:01:51.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:52.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:01:52.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:01:53.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 382252089346 -lt 382252089346 2026-02-19T13:01:53.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:53.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743432 2026-02-19T13:01:53.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:53.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:01:53.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743432 2026-02-19T13:01:53.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:53.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743432 2026-02-19T13:01:53.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743432' 2026-02-19T13:01:53.018 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 296352743432 2026-02-19T13:01:53.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:01:53.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743433 -lt 296352743432 2026-02-19T13:01:53.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:01:53.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:53.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:53.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:01:53.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:01:53.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:01:53.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:01:53.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:01:53.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:01:53.383 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:01:53.383 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:01:53.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:01:53.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:01:53.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:53.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:53.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:01:53.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:01:53.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:01:53.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3752: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-19T13:01:53.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3753: corrupt_scrub_erasure: objname=EOBJ7 2026-02-19T13:01:53.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3754: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ7 2026-02-19T13:01:53.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:01:53.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T13:01:53.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ7 2026-02-19T13:01:53.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:01:53.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:01:53.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:01:53.937 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:01:53.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:01:54.141 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:01:54.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:01:54.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:01:54.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ7 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:01:54.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: expr 7 % 2 2026-02-19T13:01:54.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3756: corrupt_scrub_erasure: local osd=1 2026-02-19T13:01:54.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3758: corrupt_scrub_erasure: case $i in 2026-02-19T13:01:54.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3803: corrupt_scrub_erasure: local payload=MAKETHISDIFFERENTFROMOTHEROBJECTS 2026-02-19T13:01:54.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3804: corrupt_scrub_erasure: echo MAKETHISDIFFERENTFROMOTHEROBJECTS 2026-02-19T13:01:54.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3805: corrupt_scrub_erasure: rados --pool ecpool put EOBJ7 td/osd-scrub-repair/DIFFERENT 2026-02-19T13:01:54.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3808: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-19T13:01:54.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:01:54.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:01:54.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:01:54.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:01:54.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-19T13:01:54.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:01:54.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:01:54.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:01:54.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:01:54.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:01:54.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:01:54.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:01:54.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:01:54.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:01:54.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:01:54.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:01:54.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-19T13:01:54.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:01:54.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:01:54.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:01:54.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:01:54.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:01:54.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ1 get-attr hinfo_key 2026-02-19T13:01:54.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:01:54.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:01:54.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:01:54.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:01:54.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:01:54.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:01:54.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:01:54.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:01:54.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:01:54.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:01:54.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:01:54.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:01:54.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:01:54.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:01:54.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:01:54.678 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:01:54.678 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:01:54.679 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:01:54.679 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:01:54.679 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:01:54.679 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:01:54.679 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:54.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:01:54.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:01:54.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:01:54.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:01:54.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:01:54.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:01:54.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:01:54.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:01:54.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:01:54.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:01:54.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:01:54.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:01:54.681 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:01:54.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:01:54.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:01:54.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:01:54.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:01:54.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:01:54.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:01:54.703 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:54.702+0000 7f62b1e40a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:54.704 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:54.702+0000 7f62b1e40a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:54.705 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:54.702+0000 7f62b1e40a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:01:54.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:01:54.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:01:54.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:01:54.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:01:54.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:01:54.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:54.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:01:54.840 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:01:54.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:54.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:01:55.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:01:55.092 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:55.090+0000 7f62b1e40a40 -1 Falling back to public interface 2026-02-19T13:01:55.675 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:01:55.674+0000 7f62b1e40a40 -1 osd.0 92 log_to_monitors true 2026-02-19T13:01:56.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:01:56.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:01:56.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:01:56.005 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:56.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:01:56.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:01:56.158 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 96 up_thru 96 down_at 93 last_clean_interval [84,92) [v2:127.0.0.1:6802/955769956,v1:127.0.0.1:6803/955769956] [v2:127.0.0.1:6804/955769956,v1:127.0.0.1:6805/955769956] exists,up 9466bb56-717c-4025-8eb0-90ced13b4bba 2026-02-19T13:01:56.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:01:56.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:01:56.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:01:56.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:01:56.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:01:56.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:01:56.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:01:56.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:01:56.160 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:01:56.160 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:01:56.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:01:56.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:01:56.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:01:56.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:01:56.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:01:56.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:01:56.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:01:56.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:01:56.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:01:56.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:01:56.431 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:01:56.431 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:01:56.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:01:56.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:56.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:01:56.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=412316860418 2026-02-19T13:01:56.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 412316860418 2026-02-19T13:01:56.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-412316860418' 2026-02-19T13:01:56.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:56.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:01:56.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=382252089348 2026-02-19T13:01:56.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 382252089348 2026-02-19T13:01:56.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-412316860418 1-382252089348' 2026-02-19T13:01:56.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:01:56.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:01:56.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743434 2026-02-19T13:01:56.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743434 2026-02-19T13:01:56.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-412316860418 1-382252089348 2-296352743434' 2026-02-19T13:01:56.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:56.658 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-412316860418 2026-02-19T13:01:56.658 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:56.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:01:56.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-412316860418 2026-02-19T13:01:56.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:56.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=412316860418 2026-02-19T13:01:56.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 412316860418' 2026-02-19T13:01:56.662 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 412316860418 2026-02-19T13:01:56.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:56.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 412316860418 2026-02-19T13:01:56.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:57.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:01:57.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:57.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 412316860418 2026-02-19T13:01:57.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:01:58.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:01:58.995 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:01:59.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 412316860418 -lt 412316860418 2026-02-19T13:01:59.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:59.156 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-382252089348 2026-02-19T13:01:59.156 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:59.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:01:59.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-382252089348 2026-02-19T13:01:59.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:59.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=382252089348 2026-02-19T13:01:59.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 382252089348' 2026-02-19T13:01:59.160 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 382252089348 2026-02-19T13:01:59.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:01:59.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 382252089348 -lt 382252089348 2026-02-19T13:01:59.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:01:59.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743434 2026-02-19T13:01:59.328 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:01:59.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:01:59.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743434 2026-02-19T13:01:59.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:01:59.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743434 2026-02-19T13:01:59.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743434' 2026-02-19T13:01:59.331 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 296352743434 2026-02-19T13:01:59.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:01:59.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743435 -lt 296352743434 2026-02-19T13:01:59.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:01:59.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:59.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:01:59.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:01:59.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:01:59.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:01:59.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:01:59.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:01:59.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:01:59.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:01:59.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:01:59.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:01:59.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:01:59.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:01:59.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:02:00.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:02:00.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:02:00.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:02:00.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3809: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-19T13:02:00.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:02:00.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:02:00.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:02:00.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:02:00.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-19T13:02:00.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:02:00.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:02:00.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:02:00.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:02:00.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:02:00.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:02:00.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:02:00.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:02:00.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:02:00.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:02:00.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:02:00.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-19T13:02:00.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:02:00.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:02:00.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:02:00.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:02:00.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:02:00.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-19T13:02:00.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:02:00.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:02:00.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:02:00.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:02:00.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:02:00.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:02:00.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:02:00.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:02:00.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:02:00.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:02:00.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:02:00.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:02:00.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:02:00.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:02:00.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:02:00.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:02:00.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:02:00.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:02:00.629 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:02:00.629 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:02:00.630 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:02:00.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:02:00.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:02:00.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:02:00.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:02:00.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:02:00.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:02:00.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:02:00.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:02:00.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:02:00.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:02:00.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:02:00.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:02:00.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:02:00.631 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:02:00.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:02:00.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:02:00.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:02:00.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:02:00.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:02:00.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:02:00.653 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:00.650+0000 7fd73091ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:02:00.654 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:00.654+0000 7fd73091ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:02:00.655 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:00.654+0000 7fd73091ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:02:00.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:02:00.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:02:00.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:02:00.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:02:00.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:02:00.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:02:00.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:02:00.788 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:02:00.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:02:00.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:02:00.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:02:01.064 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:01.062+0000 7fd73091ba40 -1 Falling back to public interface 2026-02-19T13:02:01.668 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:01.666+0000 7fd73091ba40 -1 osd.0 98 log_to_monitors true 2026-02-19T13:02:01.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:02:01.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:02:01.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:02:01.951 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:02:01.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:02:01.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:02:02.107 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 102 up_thru 102 down_at 99 last_clean_interval [96,98) [v2:127.0.0.1:6802/1274678075,v1:127.0.0.1:6803/1274678075] [v2:127.0.0.1:6804/1274678075,v1:127.0.0.1:6805/1274678075] exists,up 9466bb56-717c-4025-8eb0-90ced13b4bba 2026-02-19T13:02:02.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:02:02.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:02:02.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:02:02.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:02:02.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:02:02.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:02:02.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:02:02.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:02:02.108 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:02:02.108 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:02:02.109 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:02:02.109 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:02:02.109 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:02:02.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:02:02.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:02:02.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:02:02.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:02:02.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:02:02.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:02:02.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:02:02.381 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:02:02.381 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:02:02.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:02:02.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:02:02.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:02:02.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=438086664194 2026-02-19T13:02:02.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 438086664194 2026-02-19T13:02:02.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664194' 2026-02-19T13:02:02.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:02:02.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:02:02.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=382252089350 2026-02-19T13:02:02.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 382252089350 2026-02-19T13:02:02.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664194 1-382252089350' 2026-02-19T13:02:02.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:02:02.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:02:02.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743437 2026-02-19T13:02:02.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743437 2026-02-19T13:02:02.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664194 1-382252089350 2-296352743437' 2026-02-19T13:02:02.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:02:02.605 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-438086664194 2026-02-19T13:02:02.605 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:02:02.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:02:02.607 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-438086664194 2026-02-19T13:02:02.607 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:02:02.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=438086664194 2026-02-19T13:02:02.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 438086664194' 2026-02-19T13:02:02.609 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 438086664194 2026-02-19T13:02:02.609 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:02:02.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 438086664194 2026-02-19T13:02:02.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:02:03.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:02:03.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:02:03.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 438086664194 2026-02-19T13:02:03.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:02:04.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:02:04.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:02:05.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664194 -lt 438086664194 2026-02-19T13:02:05.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:02:05.100 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-382252089350 2026-02-19T13:02:05.100 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:02:05.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:02:05.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-382252089350 2026-02-19T13:02:05.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:02:05.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=382252089350 2026-02-19T13:02:05.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 382252089350' 2026-02-19T13:02:05.104 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 382252089350 2026-02-19T13:02:05.104 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:02:05.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 382252089350 -lt 382252089350 2026-02-19T13:02:05.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:02:05.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743437 2026-02-19T13:02:05.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:02:05.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:02:05.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743437 2026-02-19T13:02:05.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:02:05.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743437 2026-02-19T13:02:05.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743437' 2026-02-19T13:02:05.266 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 296352743437 2026-02-19T13:02:05.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:02:05.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743437 -lt 296352743437 2026-02-19T13:02:05.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:02:05.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:02:05.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:02:05.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:02:05.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:02:05.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:02:05.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:02:05.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:02:05.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:02:05.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:02:05.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:02:05.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:02:05.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:02:05.796 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:02:05.796 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:02:05.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:02:05.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:02:05.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:02:05.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3810: corrupt_scrub_erasure: rm -f td/osd-scrub-repair/hinfo 2026-02-19T13:02:05.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3816: corrupt_scrub_erasure: get_pg ecpool EOBJ0 2026-02-19T13:02:05.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T13:02:05.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=EOBJ0 2026-02-19T13:02:05.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool EOBJ0 2026-02-19T13:02:05.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:02:06.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3816: corrupt_scrub_erasure: local pg=3.0 2026-02-19T13:02:06.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3818: corrupt_scrub_erasure: pg_scrub 3.0 2026-02-19T13:02:06.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=3.0 2026-02-19T13:02:06.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 3.0 2026-02-19T13:02:06.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=3.0 2026-02-19T13:02:06.153 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:02:06.154 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:02:06.154 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:02:06.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:02:06.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:02:06.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:02:06.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:02:06.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:02:06.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:02:06.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:02:06.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:02:06.443 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:02:06.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:02:06.606 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:02:06.606 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:02:06.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:02:06.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:02:06.606 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:02:06.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=438086664195 2026-02-19T13:02:06.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 438086664195 2026-02-19T13:02:06.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664195' 2026-02-19T13:02:06.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:02:06.678 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:02:06.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=382252089352 2026-02-19T13:02:06.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 382252089352 2026-02-19T13:02:06.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664195 1-382252089352' 2026-02-19T13:02:06.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:02:06.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:02:06.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743438 2026-02-19T13:02:06.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743438 2026-02-19T13:02:06.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664195 1-382252089352 2-296352743438' 2026-02-19T13:02:06.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:02:06.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-438086664195 2026-02-19T13:02:06.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:02:06.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:02:06.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-438086664195 2026-02-19T13:02:06.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:02:06.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=438086664195 2026-02-19T13:02:06.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 438086664195' 2026-02-19T13:02:06.827 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 438086664195 2026-02-19T13:02:06.827 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:02:06.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664194 -lt 438086664195 2026-02-19T13:02:06.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:02:07.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:02:07.987 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:02:08.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664194 -lt 438086664195 2026-02-19T13:02:08.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:02:09.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:02:09.148 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:02:09.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664196 -lt 438086664195 2026-02-19T13:02:09.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:02:09.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-382252089352 2026-02-19T13:02:09.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:02:09.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:02:09.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-382252089352 2026-02-19T13:02:09.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:02:09.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=382252089352 2026-02-19T13:02:09.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 382252089352' 2026-02-19T13:02:09.316 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 382252089352 2026-02-19T13:02:09.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:02:09.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 382252089352 -lt 382252089352 2026-02-19T13:02:09.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:02:09.488 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743438 2026-02-19T13:02:09.488 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:02:09.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:02:09.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743438 2026-02-19T13:02:09.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:02:09.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743438 2026-02-19T13:02:09.492 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 296352743438 2026-02-19T13:02:09.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743438' 2026-02-19T13:02:09.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:02:09.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743439 -lt 296352743438 2026-02-19T13:02:09.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:02:09.649 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 3.0 loop 0 2026-02-19T13:02:09.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-19T13:02:09.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 3.0 2026-02-19T13:02:09.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=3.0 2026-02-19T13:02:09.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:02:09.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 3.0 query 2026-02-19T13:02:09.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:02:09.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-19T13:02:09.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:02:09.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:02:09.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:02:09.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 3.0 2026-02-19T13:02:09.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:09.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:02:09.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:09.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:02:09.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:09.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 3.0 2026-02-19T13:02:10.030 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0s0 on osd.1 to scrub 2026-02-19T13:02:10.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 3.0 2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:10.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:02:10.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:10.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:02:10.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:02:10.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:10.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:02:10.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:10.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:02:10.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:10.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:02:10.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:00:54.267671+0000 '>' 2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:10.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:02:11.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:02:11.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:11.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:02:11.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:11.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:02:11.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:11.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:02:11.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:00:54.267671+0000 '>' 2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:11.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:02:12.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:02:12.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:12.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:02:12.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:12.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:02:12.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:12.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:02:12.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:00:54.267671+0000 '>' 2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:12.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:02:13.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:02:13.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:13.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:02:13.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:13.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:02:13.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:13.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:02:13.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:00:54.267671+0000 '>' 2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:13.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:02:14.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:02:14.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:14.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:02:14.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:14.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:02:14.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:14.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:02:14.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:00:54.267671+0000 '>' 2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:14.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:02:15.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:02:15.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:15.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:02:15.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:15.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:02:15.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:02:15.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:15.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:00:54.267671+0000 '>' 2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:15.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:02:16.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:02:16.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:16.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:02:16.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:16.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:02:16.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:16.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:02:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:02:10.282749+0000 '>' 2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:02:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3820: corrupt_scrub_erasure: rados list-inconsistent-pg ecpool 2026-02-19T13:02:17.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3822: corrupt_scrub_erasure: jq '. | length' td/osd-scrub-repair/json 2026-02-19T13:02:17.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3822: corrupt_scrub_erasure: test 1 = 1 2026-02-19T13:02:17.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3824: corrupt_scrub_erasure: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-19T13:02:17.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3824: corrupt_scrub_erasure: test 3.0 = 3.0 2026-02-19T13:02:17.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3826: corrupt_scrub_erasure: rados list-inconsistent-obj 3.0 2026-02-19T13:02:17.214 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3828: corrupt_scrub_erasure: jq .epoch td/osd-scrub-repair/json 2026-02-19T13:02:17.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3828: corrupt_scrub_erasure: epoch=102 2026-02-19T13:02:17.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3830: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-19T13:02:17.230 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:02:17.230 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:02:17.230 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:02:17.230 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:02:17.230 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:02:17.230 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:02:17.230 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:02:17.230 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:02:17.230 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:02:17.230 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:02:17.230 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:02:17.231 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-19T13:02:17.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3830: corrupt_scrub_erasure: jq .inconsistents 2026-02-19T13:02:17.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3830: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:02:17.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4458: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-19T13:02:17.247 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:02:17.247 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:02:17.247 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:02:17.247 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:02:17.247 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:02:17.248 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:02:17.248 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:02:17.248 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:02:17.248 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:02:17.248 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:02:17.248 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:02:17.248 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-19T13:02:17.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4458: corrupt_scrub_erasure: jq .inconsistents 2026-02-19T13:02:17.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4458: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:02:17.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4459: corrupt_scrub_erasure: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-19T13:02:17.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-19T13:02:17.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4460: corrupt_scrub_erasure: test no = yes 2026-02-19T13:02:17.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4465: corrupt_scrub_erasure: test '' = yes 2026-02-19T13:02:17.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4470: corrupt_scrub_erasure: pg_deep_scrub 3.0 2026-02-19T13:02:17.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_deep_scrub: local pgid=3.0 2026-02-19T13:02:17.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_deep_scrub: wait_for_pg_clean 3.0 2026-02-19T13:02:17.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=3.0 2026-02-19T13:02:17.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:02:17.275 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:02:17.275 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:02:17.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:02:17.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:02:17.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:02:17.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:02:17.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:02:17.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:02:17.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:02:17.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:02:17.575 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:02:17.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:02:17.729 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:02:17.729 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:02:17.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:02:17.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:02:17.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:02:17.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=438086664199 2026-02-19T13:02:17.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 438086664199 2026-02-19T13:02:17.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664199' 2026-02-19T13:02:17.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:02:17.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:02:17.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=382252089355 2026-02-19T13:02:17.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 382252089355 2026-02-19T13:02:17.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664199 1-382252089355' 2026-02-19T13:02:17.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:02:17.876 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:02:17.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=296352743442 2026-02-19T13:02:17.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 296352743442 2026-02-19T13:02:17.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664199 1-382252089355 2-296352743442' 2026-02-19T13:02:17.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:02:17.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-438086664199 2026-02-19T13:02:17.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:02:17.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:02:17.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-438086664199 2026-02-19T13:02:17.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:02:17.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=438086664199 2026-02-19T13:02:17.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 438086664199' 2026-02-19T13:02:17.952 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 438086664199 2026-02-19T13:02:17.952 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:02:18.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664197 -lt 438086664199 2026-02-19T13:02:18.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:02:19.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:02:19.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:02:19.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664199 -lt 438086664199 2026-02-19T13:02:19.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:02:19.274 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-382252089355 2026-02-19T13:02:19.274 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:02:19.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:02:19.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-382252089355 2026-02-19T13:02:19.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:02:19.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=382252089355 2026-02-19T13:02:19.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 382252089355' 2026-02-19T13:02:19.278 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 382252089355 2026-02-19T13:02:19.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:02:19.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 382252089355 -lt 382252089355 2026-02-19T13:02:19.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:02:19.443 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-296352743442 2026-02-19T13:02:19.443 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:02:19.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:02:19.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-296352743442 2026-02-19T13:02:19.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:02:19.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=296352743442 2026-02-19T13:02:19.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 296352743442' 2026-02-19T13:02:19.447 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 296352743442 2026-02-19T13:02:19.447 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:02:19.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 296352743442 -lt 296352743442 2026-02-19T13:02:19.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:02:19.616 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 3.0 loop 0 2026-02-19T13:02:19.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-19T13:02:19.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 3.0 2026-02-19T13:02:19.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=3.0 2026-02-19T13:02:19.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:02:19.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 3.0 query 2026-02-19T13:02:19.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:02:19.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean+inconsistent 2026-02-19T13:02:19.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean+inconsistent == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:02:19.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:02:19.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:02:19.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:02:19.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:19.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:02:19.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:19.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:02:19.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: local last_scrub=2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:19.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_deep_scrub: ceph pg deep-scrub 3.0 2026-02-19T13:02:19.999 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0s0 on osd.1 to deep-scrub 2026-02-19T13:02:20.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_deep_scrub: wait_for_scrub 3.0 2026-02-19T13:00:54.267671+0000 last_deep_scrub_stamp 2026-02-19T13:02:20.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:02:20.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:20.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-19T13:02:20.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:02:20.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:20.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:02:20.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:20.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:02:20.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:20.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:02:20.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:00:54.267671+0000 '>' 2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:20.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:02:21.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:02:21.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:21.172 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:02:21.172 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:21.172 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:02:21.172 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:21.172 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:02:21.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:00:54.267671+0000 '>' 2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:21.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:02:22.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:02:22.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:22.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:02:22.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:22.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:02:22.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:02:22.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:22.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:00:54.267671+0000 '>' 2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:22.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:02:23.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:02:23.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:23.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:02:23.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:23.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:02:23.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:23.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:02:23.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:00:54.267671+0000 '>' 2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:23.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:02:24.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:02:24.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:24.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:02:24.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:24.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:02:24.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:24.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:02:24.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:00:54.267671+0000 '>' 2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:24.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:02:25.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:02:25.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:25.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:02:25.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:25.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:02:25.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:25.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:02:25.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:00:54.267671+0000 '>' 2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:25.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:02:26.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:02:26.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:26.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:02:26.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:26.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:02:26.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:02:26.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:27.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:02:20.340569+0000 '>' 2026-02-19T13:00:54.267671+0000 2026-02-19T13:02:27.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:02:27.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4472: corrupt_scrub_erasure: rados list-inconsistent-pg ecpool 2026-02-19T13:02:27.155 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4474: corrupt_scrub_erasure: jq '. | length' td/osd-scrub-repair/json 2026-02-19T13:02:27.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4474: corrupt_scrub_erasure: test 1 = 1 2026-02-19T13:02:27.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4476: corrupt_scrub_erasure: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-19T13:02:27.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4476: corrupt_scrub_erasure: test 3.0 = 3.0 2026-02-19T13:02:27.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4478: corrupt_scrub_erasure: rados list-inconsistent-obj 3.0 2026-02-19T13:02:27.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4479: corrupt_scrub_erasure: cp td/osd-scrub-repair/json /tmp/rrr6.json 2026-02-19T13:02:27.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4481: corrupt_scrub_erasure: jq .epoch td/osd-scrub-repair/json 2026-02-19T13:02:27.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4481: corrupt_scrub_erasure: epoch=102 2026-02-19T13:02:27.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4483: corrupt_scrub_erasure: '[' true = true ']' 2026-02-19T13:02:27.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4485: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-19T13:02:27.208 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:02:27.208 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:02:27.209 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:02:27.209 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:02:27.209 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:02:27.209 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:02:27.209 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:02:27.209 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:02:27.209 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:02:27.209 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:02:27.209 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:02:27.209 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-19T13:02:27.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4485: corrupt_scrub_erasure: jq .inconsistents 2026-02-19T13:02:27.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4485: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:02:27.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5890: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-19T13:02:27.228 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:02:27.228 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:02:27.228 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:02:27.228 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:02:27.228 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:02:27.228 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:02:27.228 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:02:27.228 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:02:27.228 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:02:27.228 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:02:27.228 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:02:27.228 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-19T13:02:27.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5890: corrupt_scrub_erasure: jq .inconsistents 2026-02-19T13:02:27.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5890: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:02:27.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: corrupt_scrub_erasure: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-19T13:02:27.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-19T13:02:27.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5892: corrupt_scrub_erasure: test no = yes 2026-02-19T13:02:27.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5903: corrupt_scrub_erasure: test '' = yes 2026-02-19T13:02:27.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5908: corrupt_scrub_erasure: ceph osd pool rm ecpool ecpool --yes-i-really-really-mean-it 2026-02-19T13:02:27.455 INFO:tasks.workunit.client.0.trial144.stderr:pool 'ecpool' removed 2026-02-19T13:02:27.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T13:02:27.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:02:27.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:02:27.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:02:27.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:02:27.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:02:27.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:02:27.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:02:27.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:02:27.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:02:27.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:02:27.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:02:27.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:02:27.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:02:27.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:02:27.583 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:02:27.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:02:27.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:02:27.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:02:27.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:02:27.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:02:27.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:02:27.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:02:27.611 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:02:27.611 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:02:27.611 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:02:27.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:02:27.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:02:27.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:02:27.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T13:02:27.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T13:02:27.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T13:02:27.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T13:02:27.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:02:27.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:02:27.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:02:27.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:02:27.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:02:27.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:02:27.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:02:27.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:02:27.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:02:27.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:02:27.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:02:27.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:02:27.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:02:27.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:02:27.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:02:27.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:02:27.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:02:27.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:02:27.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:02:27.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:02:27.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:02:27.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:02:27.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:02:27.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:02:27.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:02:27.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:02:27.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:02:27.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:02:27.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T13:02:27.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:02:27.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:02:27.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:02:27.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T13:02:27.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:02:27.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:02:27.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:02:27.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T13:02:27.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_scrub_replicated td/osd-scrub-repair 2026-02-19T13:02:27.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1050: TEST_corrupt_scrub_replicated: local dir=td/osd-scrub-repair 2026-02-19T13:02:27.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1051: TEST_corrupt_scrub_replicated: local poolname=csr_pool 2026-02-19T13:02:27.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1052: TEST_corrupt_scrub_replicated: local total_objs=19 2026-02-19T13:02:27.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1054: TEST_corrupt_scrub_replicated: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-19T13:02:27.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T13:02:27.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:02:27.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:02:27.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:02:27.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T13:02:27.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-19T13:02:27.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:02:27.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:02:27.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:02:27.656 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:02:27.656 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:02:27.656 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:02:27.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:02:27.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-19T13:02:27.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:02:27.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:02:27.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:02:27.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:02:27.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:02:27.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:02:27.695 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:02:27.696 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:02:27.696 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:02:27.696 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:02:27.696 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:02:27.696 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:02:27.696 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:02:27.697 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:02:27.697 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T13:02:27.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:02:27.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:02:27.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:02:27.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:02:27.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:02:27.752 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:02:27.752 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:02:27.752 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:02:27.752 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:02:27.752 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:02:27.752 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:02:27.753 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:02:27.753 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:02:27.753 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T13:02:27.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1055: TEST_corrupt_scrub_replicated: run_mgr td/osd-scrub-repair x 2026-02-19T13:02:27.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T13:02:27.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:02:27.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:02:27.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:02:27.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T13:02:27.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:02:27.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:02:27.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:02:27.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:02:27.901 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:02:27.901 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:02:27.901 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:02:27.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:02:27.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:02:27.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:02:27.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1056: TEST_corrupt_scrub_replicated: run_osd td/osd-scrub-repair 0 2026-02-19T13:02:27.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:02:27.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:02:27.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:02:27.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:02:27.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:02:27.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:02:27.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:02:27.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:02:27.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:02:27.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:02:27.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:02:27.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:02:27.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:02:27.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:02:27.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:02:27.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:02:27.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:02:27.924 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:02:27.924 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:02:27.924 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:02:27.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:02:27.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:02:27.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:02:27.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:02:27.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:02:27.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:02:27.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:02:27.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:02:27.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:02:27.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:02:27.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:02:27.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:02:27.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:02:27.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:02:27.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:02:27.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0250c601-08cf-4db7-8be3-271bfebc9558 2026-02-19T13:02:27.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0250c601-08cf-4db7-8be3-271bfebc9558' 2026-02-19T13:02:27.927 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 0250c601-08cf-4db7-8be3-271bfebc9558 2026-02-19T13:02:27.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:02:27.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDjCZdpZV7vNxAAs6r3/eRsEq+AL+omXyJWng== 2026-02-19T13:02:27.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDjCZdpZV7vNxAAs6r3/eRsEq+AL+omXyJWng=="}' 2026-02-19T13:02:27.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0250c601-08cf-4db7-8be3-271bfebc9558 -i td/osd-scrub-repair/0/new.json 2026-02-19T13:02:28.024 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:02:28.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T13:02:28.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDjCZdpZV7vNxAAs6r3/eRsEq+AL+omXyJWng== --osd-uuid 0250c601-08cf-4db7-8be3-271bfebc9558 2026-02-19T13:02:28.049 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:28.045+0000 7f71937aaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:02:28.050 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:28.049+0000 7f71937aaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:02:28.051 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:28.049+0000 7f71937aaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:02:28.052 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:28.049+0000 7f71937aaa40 -1 bdev(0x556e18f1ec00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:02:28.052 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:28.049+0000 7f71937aaa40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T13:02:28.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T13:02:28.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:02:28.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:02:28.795 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:02:28.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:02:29.002 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:02:29.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:02:29.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:02:29.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:02:29.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:02:29.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:02:29.020 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:29.017+0000 7f2d1396aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:02:29.021 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:29.017+0000 7f2d1396aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:02:29.022 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:29.017+0000 7f2d1396aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:02:29.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:02:29.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:02:29.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:02:29.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:02:29.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:02:29.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:02:29.167 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:02:29.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:02:29.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:02:29.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:02:29.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:02:29.452 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:29.449+0000 7f2d1396aa40 -1 Falling back to public interface 2026-02-19T13:02:30.068 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:30.065+0000 7f2d1396aa40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:02:30.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:02:30.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:02:30.326 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:02:30.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:02:30.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:02:30.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:02:30.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:02:31.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:02:31.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:02:31.489 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:02:31.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:02:31.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:02:31.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:02:31.648 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3791501686,v1:127.0.0.1:6803/3791501686] [v2:127.0.0.1:6804/3791501686,v1:127.0.0.1:6805/3791501686] exists,up 0250c601-08cf-4db7-8be3-271bfebc9558 2026-02-19T13:02:31.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1057: TEST_corrupt_scrub_replicated: run_osd td/osd-scrub-repair 1 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:02:31.649 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:02:31.650 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:02:31.650 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:02:31.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:02:31.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:02:31.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:02:31.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:02:31.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:02:31.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:02:31.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:02:31.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:02:31.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:02:31.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:02:31.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:02:31.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:02:31.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:02:31.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:02:31.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:02:31.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:02:31.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 c7a4d5dd-9169-4e8d-826b-e9eca7704cb3' 2026-02-19T13:02:31.653 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:02:31.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:02:31.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDnCZdpIE2vJxAA06PW3rLgTKHGkqlifKuwYQ== 2026-02-19T13:02:31.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDnCZdpIE2vJxAA06PW3rLgTKHGkqlifKuwYQ=="}' 2026-02-19T13:02:31.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 -i td/osd-scrub-repair/1/new.json 2026-02-19T13:02:31.828 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:02:31.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T13:02:31.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDnCZdpIE2vJxAA06PW3rLgTKHGkqlifKuwYQ== --osd-uuid c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:02:31.865 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:31.861+0000 7f315f318a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:02:31.867 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:31.865+0000 7f315f318a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:02:31.868 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:31.865+0000 7f315f318a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:02:31.868 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:31.865+0000 7f315f318a40 -1 bdev(0x55beece9ec00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:02:31.868 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:31.865+0000 7f315f318a40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T13:02:32.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T13:02:32.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:02:32.483 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:02:32.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:02:32.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:02:32.689 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:02:32.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:02:32.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:02:32.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:02:32.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:02:32.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:02:32.714 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:32.709+0000 7f1fe61afa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:02:32.716 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:32.713+0000 7f1fe61afa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:02:32.717 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:32.713+0000 7f1fe61afa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:02:32.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:02:32.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:02:32.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:02:32.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:02:32.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:02:32.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:02:32.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:02:32.847 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:02:32.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:02:32.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:02:33.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:02:33.108 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:33.105+0000 7f1fe61afa40 -1 Falling back to public interface 2026-02-19T13:02:33.741 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:02:33.737+0000 7f1fe61afa40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:02:34.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:02:34.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:02:34.007 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:02:34.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:02:34.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:02:34.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:02:34.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:02:35.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:02:35.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:02:35.172 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:02:35.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:02:35.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:02:35.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:02:35.331 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3146040721,v1:127.0.0.1:6811/3146040721] [v2:127.0.0.1:6812/3146040721,v1:127.0.0.1:6813/3146040721] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:02:35.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:02:35.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:02:35.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:02:35.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1058: TEST_corrupt_scrub_replicated: create_rbd_pool 2026-02-19T13:02:35.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T13:02:35.484 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T13:02:35.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T13:02:35.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:02:35.715 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T13:02:35.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:02:36.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T13:02:37.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1059: TEST_corrupt_scrub_replicated: wait_for_clean 2026-02-19T13:02:37.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:02:37.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:02:37.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:02:37.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:02:37.011 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:02:37.011 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:02:37.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:02:37.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:02:37.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:02:37.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:02:37.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:02:37.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:02:37.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:02:37.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:02:37.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:02:37.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:02:37.288 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:02:37.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:02:37.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:02:37.289 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:02:37.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T13:02:37.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T13:02:37.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T13:02:37.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:02:37.365 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:02:37.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672962 2026-02-19T13:02:37.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672962 2026-02-19T13:02:37.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-42949672962' 2026-02-19T13:02:37.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:02:37.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T13:02:37.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:02:37.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:02:37.440 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T13:02:37.440 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:02:37.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T13:02:37.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T13:02:37.440 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T13:02:37.441 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:02:37.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T13:02:37.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:02:38.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:02:38.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:02:38.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-19T13:02:38.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:02:38.760 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672962 2026-02-19T13:02:38.760 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:02:38.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:02:38.762 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672962 2026-02-19T13:02:38.762 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:02:38.764 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672962 2026-02-19T13:02:38.764 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672962' 2026-02-19T13:02:38.764 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672962 2026-02-19T13:02:38.765 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:02:38.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672962 -lt 42949672962 2026-02-19T13:02:38.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:02:38.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:02:38.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:02:39.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T13:02:39.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:02:39.121 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:02:39.121 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:02:39.121 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:02:39.121 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:02:39.121 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:02:39.121 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:02:39.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T13:02:39.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:02:39.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:02:39.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:02:39.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T13:02:39.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:02:39.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:02:39.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1061: TEST_corrupt_scrub_replicated: create_pool foo 1 2026-02-19T13:02:39.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 1 2026-02-19T13:02:39.682 INFO:tasks.workunit.client.0.trial144.stderr:pool 'foo' created 2026-02-19T13:02:39.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:02:40.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1062: TEST_corrupt_scrub_replicated: create_pool csr_pool 1 1 2026-02-19T13:02:40.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create csr_pool 1 1 2026-02-19T13:02:40.902 INFO:tasks.workunit.client.0.trial144.stderr:pool 'csr_pool' created 2026-02-19T13:02:40.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:02:41.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1063: TEST_corrupt_scrub_replicated: wait_for_clean 2026-02-19T13:02:41.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:02:41.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:02:41.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:02:41.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:02:41.915 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:02:41.915 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:02:41.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:02:41.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:02:41.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:02:42.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:02:42.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:02:42.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:02:42.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:02:42.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:02:42.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:02:42.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:02:42.178 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:02:42.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:02:42.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:02:42.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:02:42.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-19T13:02:42.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-19T13:02:42.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485' 2026-02-19T13:02:42.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:02:42.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:02:42.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672964 2026-02-19T13:02:42.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672964 2026-02-19T13:02:42.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-19T13:02:42.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:02:42.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-19T13:02:42.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:02:42.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:02:42.328 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836485 2026-02-19T13:02:42.328 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:02:42.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836485 2026-02-19T13:02:42.329 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836485 2026-02-19T13:02:42.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-19T13:02:42.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:02:42.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-19T13:02:42.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:02:42.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672964 2026-02-19T13:02:42.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:02:42.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:02:42.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672964 2026-02-19T13:02:42.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:02:42.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672964 2026-02-19T13:02:42.506 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672964 2026-02-19T13:02:42.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-19T13:02:42.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:02:42.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-19T13:02:42.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:02:42.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:02:42.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:02:42.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:02:42.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:02:42.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:02:42.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:02:42.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:02:42.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:02:42.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:02:42.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:02:43.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:02:43.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:02:43.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:02:43.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:02:43.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:02:43.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:02:43.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:02:43.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: seq 1 19 2026-02-19T13:02:43.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:43.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ1 2026-02-19T13:02:43.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ1 2026-02-19T13:02:43.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:43.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:43.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ1 2026-02-19T13:02:43.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:43.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:43.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:43.408 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:43.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:43.613 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:43.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:43.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:43.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:43.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ1 hdr-ROBJ1 2026-02-19T13:02:43.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ1 key-ROBJ1 val-ROBJ1 2026-02-19T13:02:43.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:43.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ2 2026-02-19T13:02:43.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ2 2026-02-19T13:02:43.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:43.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:43.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ2 2026-02-19T13:02:43.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:43.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:43.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:43.899 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:43.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:44.104 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:44.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:44.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:44.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ2 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:44.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ2 hdr-ROBJ2 2026-02-19T13:02:44.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ2 key-ROBJ2 val-ROBJ2 2026-02-19T13:02:44.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:44.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ3 2026-02-19T13:02:44.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ3 2026-02-19T13:02:44.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:44.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:44.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ3 2026-02-19T13:02:44.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:44.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:44.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:44.398 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:44.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:44.602 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:44.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:44.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:44.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ3 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:44.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ3 hdr-ROBJ3 2026-02-19T13:02:44.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ3 key-ROBJ3 val-ROBJ3 2026-02-19T13:02:44.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:44.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ4 2026-02-19T13:02:44.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ4 2026-02-19T13:02:44.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:44.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:44.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ4 2026-02-19T13:02:44.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:44.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:44.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:44.901 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:44.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:45.106 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:45.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:45.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:45.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ4 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:45.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ4 hdr-ROBJ4 2026-02-19T13:02:45.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ4 key-ROBJ4 val-ROBJ4 2026-02-19T13:02:45.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:45.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ5 2026-02-19T13:02:45.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ5 2026-02-19T13:02:45.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:45.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:45.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ5 2026-02-19T13:02:45.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:45.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:45.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:45.404 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:45.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:45.609 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:45.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:45.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:45.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ5 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:45.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ5 hdr-ROBJ5 2026-02-19T13:02:45.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ5 key-ROBJ5 val-ROBJ5 2026-02-19T13:02:45.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:45.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ6 2026-02-19T13:02:45.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ6 2026-02-19T13:02:45.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:45.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:45.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ6 2026-02-19T13:02:45.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:45.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:45.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:45.901 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:45.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:46.106 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:46.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:46.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:46.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ6 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:46.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ6 hdr-ROBJ6 2026-02-19T13:02:46.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ6 key-ROBJ6 val-ROBJ6 2026-02-19T13:02:46.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:46.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ7 2026-02-19T13:02:46.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ7 2026-02-19T13:02:46.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:46.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:46.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ7 2026-02-19T13:02:46.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:46.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:46.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:46.399 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:46.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:46.603 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:46.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:46.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:46.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ7 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:46.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ7 hdr-ROBJ7 2026-02-19T13:02:46.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ7 key-ROBJ7 val-ROBJ7 2026-02-19T13:02:46.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:46.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ8 2026-02-19T13:02:46.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ8 2026-02-19T13:02:46.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:46.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:46.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ8 2026-02-19T13:02:46.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:46.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:46.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:46.902 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:46.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:47.106 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:47.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:47.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:47.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ8 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:47.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ8 hdr-ROBJ8 2026-02-19T13:02:47.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ8 key-ROBJ8 val-ROBJ8 2026-02-19T13:02:47.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:47.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ9 2026-02-19T13:02:47.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ9 2026-02-19T13:02:47.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:47.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:47.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ9 2026-02-19T13:02:47.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:47.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:47.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:47.405 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:47.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:47.608 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:47.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:47.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:47.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ9 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:47.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ9 hdr-ROBJ9 2026-02-19T13:02:47.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ9 key-ROBJ9 val-ROBJ9 2026-02-19T13:02:47.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:47.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ10 2026-02-19T13:02:47.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ10 2026-02-19T13:02:47.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:47.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:47.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ10 2026-02-19T13:02:47.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:47.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:47.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:47.896 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:47.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:48.100 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:48.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:48.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:48.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ10 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:48.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ10 hdr-ROBJ10 2026-02-19T13:02:48.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ10 key-ROBJ10 val-ROBJ10 2026-02-19T13:02:48.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:48.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ11 2026-02-19T13:02:48.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ11 2026-02-19T13:02:48.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:48.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:48.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ11 2026-02-19T13:02:48.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:48.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:48.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:48.394 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:48.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:48.598 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:48.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:48.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:48.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ11 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:48.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ11 hdr-ROBJ11 2026-02-19T13:02:48.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ11 key-ROBJ11 val-ROBJ11 2026-02-19T13:02:48.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:48.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ12 2026-02-19T13:02:48.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ12 2026-02-19T13:02:48.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:48.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:48.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ12 2026-02-19T13:02:48.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:48.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:48.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:48.887 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:48.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:49.091 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:49.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:49.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:49.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ12 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:49.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ12 hdr-ROBJ12 2026-02-19T13:02:49.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ12 key-ROBJ12 val-ROBJ12 2026-02-19T13:02:49.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:49.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ13 2026-02-19T13:02:49.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ13 2026-02-19T13:02:49.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:49.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:49.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ13 2026-02-19T13:02:49.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:49.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:49.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:49.378 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:49.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:49.582 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:49.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:49.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:49.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ13 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:49.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ13 hdr-ROBJ13 2026-02-19T13:02:49.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ13 key-ROBJ13 val-ROBJ13 2026-02-19T13:02:49.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:49.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ14 2026-02-19T13:02:49.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ14 2026-02-19T13:02:49.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:49.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:49.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ14 2026-02-19T13:02:49.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:49.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:49.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:49.876 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:49.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:50.081 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:50.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:50.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:50.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ14 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:50.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ14 hdr-ROBJ14 2026-02-19T13:02:50.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ14 key-ROBJ14 val-ROBJ14 2026-02-19T13:02:50.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:50.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ15 2026-02-19T13:02:50.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ15 2026-02-19T13:02:50.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:50.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:50.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ15 2026-02-19T13:02:50.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:50.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:50.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:50.370 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:50.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:50.575 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:50.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:50.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:50.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ15 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:50.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ15 hdr-ROBJ15 2026-02-19T13:02:50.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ15 key-ROBJ15 val-ROBJ15 2026-02-19T13:02:50.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:50.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ16 2026-02-19T13:02:50.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ16 2026-02-19T13:02:50.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:50.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:50.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ16 2026-02-19T13:02:50.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:50.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:50.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:50.865 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:50.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:51.070 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ16 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:51.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ16 hdr-ROBJ16 2026-02-19T13:02:51.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ16 key-ROBJ16 val-ROBJ16 2026-02-19T13:02:51.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:51.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ17 2026-02-19T13:02:51.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ17 2026-02-19T13:02:51.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:51.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:51.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ17 2026-02-19T13:02:51.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:51.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:51.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:51.358 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:51.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:51.562 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:51.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:51.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:51.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ17 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:51.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ17 hdr-ROBJ17 2026-02-19T13:02:51.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ17 key-ROBJ17 val-ROBJ17 2026-02-19T13:02:51.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:51.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ18 2026-02-19T13:02:51.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ18 2026-02-19T13:02:51.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:51.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:51.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ18 2026-02-19T13:02:51.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:51.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:51.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:51.861 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:51.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:52.066 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:52.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:52.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:52.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ18 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:52.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ18 hdr-ROBJ18 2026-02-19T13:02:52.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ18 key-ROBJ18 val-ROBJ18 2026-02-19T13:02:52.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1065: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:02:52.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1066: TEST_corrupt_scrub_replicated: objname=ROBJ19 2026-02-19T13:02:52.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ19 2026-02-19T13:02:52.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:02:52.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:02:52.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ19 2026-02-19T13:02:52.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:02:52.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:02:52.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:02:52.356 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:02:52.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:02:52.561 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:02:52.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:02:52.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:02:52.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ19 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:02:52.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ19 hdr-ROBJ19 2026-02-19T13:02:52.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1070: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ19 key-ROBJ19 val-ROBJ19 2026-02-19T13:02:52.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1074: TEST_corrupt_scrub_replicated: dd if=/dev/zero of=td/osd-scrub-repair/new.ROBJ19 bs=1024 count=1025 2026-02-19T13:02:52.656 INFO:tasks.workunit.client.0.trial144.stderr:1025+0 records in 2026-02-19T13:02:52.656 INFO:tasks.workunit.client.0.trial144.stderr:1025+0 records out 2026-02-19T13:02:52.656 INFO:tasks.workunit.client.0.trial144.stderr:1049600 bytes (1.0 MB, 1.0 MiB) copied, 0.0040669 s, 258 MB/s 2026-02-19T13:02:52.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1075: TEST_corrupt_scrub_replicated: rados --pool csr_pool put ROBJ19 td/osd-scrub-repair/new.ROBJ19 2026-02-19T13:02:52.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1076: TEST_corrupt_scrub_replicated: rm -f td/osd-scrub-repair/new.ROBJ19 2026-02-19T13:02:52.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1078: TEST_corrupt_scrub_replicated: get_pg csr_pool ROBJ0 2026-02-19T13:02:52.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=csr_pool 2026-02-19T13:02:52.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=ROBJ0 2026-02-19T13:02:52.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map csr_pool ROBJ0 2026-02-19T13:02:52.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:02:52.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1078: TEST_corrupt_scrub_replicated: local pg=3.0 2026-02-19T13:02:52.849 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1079: TEST_corrupt_scrub_replicated: get_primary csr_pool ROBJ0 2026-02-19T13:02:52.849 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=csr_pool 2026-02-19T13:02:52.849 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=ROBJ0 2026-02-19T13:02:52.849 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map csr_pool ROBJ0 2026-02-19T13:02:52.849 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:02:53.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1079: TEST_corrupt_scrub_replicated: local primary=1 2026-02-19T13:02:53.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1082: TEST_corrupt_scrub_replicated: get_asok_path osd.0 2026-02-19T13:02:53.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:02:53.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:02:53.012 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:02:53.012 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:02:53.012 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:02:53.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.0.asok 2026-02-19T13:02:53.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1082: TEST_corrupt_scrub_replicated: CEPH_ARGS= 2026-02-19T13:02:53.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1082: TEST_corrupt_scrub_replicated: ceph daemon /tmp/ceph-asok.35818/ceph-osd.0.asok config set osd_deep_scrub_update_digest_min_age 0 2026-02-19T13:02:53.074 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:02:53.075 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_deep_scrub_update_digest_min_age = '' (not observed, change may require restart) " 2026-02-19T13:02:53.075 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:02:53.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1084: TEST_corrupt_scrub_replicated: get_asok_path osd.1 2026-02-19T13:02:53.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:02:53.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:02:53.081 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:02:53.081 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:02:53.081 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:02:53.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T13:02:53.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1084: TEST_corrupt_scrub_replicated: CEPH_ARGS= 2026-02-19T13:02:53.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1084: TEST_corrupt_scrub_replicated: ceph daemon /tmp/ceph-asok.35818/ceph-osd.1.asok config set osd_deep_scrub_update_digest_min_age 0 2026-02-19T13:02:53.135 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:02:53.135 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_deep_scrub_update_digest_min_age = '' (not observed, change may require restart) " 2026-02-19T13:02:53.135 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:02:53.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1086: TEST_corrupt_scrub_replicated: pg_deep_scrub 3.0 2026-02-19T13:02:53.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_deep_scrub: local pgid=3.0 2026-02-19T13:02:53.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_deep_scrub: wait_for_pg_clean 3.0 2026-02-19T13:02:53.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=3.0 2026-02-19T13:02:53.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:02:53.143 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:02:53.143 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:02:53.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:02:53.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:02:53.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:02:53.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:02:53.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:02:53.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:02:53.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:02:53.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:02:53.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:02:53.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:02:53.588 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:02:53.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:02:53.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:02:53.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:02:53.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836488 2026-02-19T13:02:53.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836488 2026-02-19T13:02:53.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488' 2026-02-19T13:02:53.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:02:53.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:02:53.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672967 2026-02-19T13:02:53.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672967 2026-02-19T13:02:53.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-42949672967' 2026-02-19T13:02:53.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:02:53.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836488 2026-02-19T13:02:53.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:02:53.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:02:53.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836488 2026-02-19T13:02:53.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:02:53.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836488 2026-02-19T13:02:53.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-19T13:02:53.743 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836488 2026-02-19T13:02:53.744 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:02:53.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836488 2026-02-19T13:02:53.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:02:54.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:02:54.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:02:55.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-19T13:02:55.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:02:55.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672967 2026-02-19T13:02:55.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:02:55.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:02:55.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672967 2026-02-19T13:02:55.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:02:55.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672967 2026-02-19T13:02:55.068 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672967 2026-02-19T13:02:55.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672967' 2026-02-19T13:02:55.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:02:55.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672967 -lt 42949672967 2026-02-19T13:02:55.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:02:55.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-19T13:02:55.238 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 3.0 loop 0 2026-02-19T13:02:55.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 3.0 2026-02-19T13:02:55.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=3.0 2026-02-19T13:02:55.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:02:55.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 3.0 query 2026-02-19T13:02:55.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:02:55.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-19T13:02:55.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:02:55.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:02:55.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:02:55.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:02:55.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:55.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:02:55.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:55.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:02:55.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: local last_scrub=2026-02-19T13:02:40.902307+0000 2026-02-19T13:02:55.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_deep_scrub: ceph pg deep-scrub 3.0 2026-02-19T13:02:55.615 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2026-02-19T13:02:55.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_deep_scrub: wait_for_scrub 3.0 2026-02-19T13:02:40.902307+0000 last_deep_scrub_stamp 2026-02-19T13:02:55.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:02:55.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:02:40.902307+0000 2026-02-19T13:02:55.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-19T13:02:55.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:02:55.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:55.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:02:55.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:55.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:02:55.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:55.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:02:55.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:02:40.902307+0000 '>' 2026-02-19T13:02:40.902307+0000 2026-02-19T13:02:55.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:02:56.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:02:56.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:56.786 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:02:56.786 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:56.786 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:02:56.787 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:56.787 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:02:56.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:02:40.902307+0000 '>' 2026-02-19T13:02:40.902307+0000 2026-02-19T13:02:56.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:02:57.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:02:57.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:57.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:02:57.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:57.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:02:57.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:57.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:02:58.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:02:40.902307+0000 '>' 2026-02-19T13:02:40.902307+0000 2026-02-19T13:02:58.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:02:59.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:02:59.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:02:59.107 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:02:59.107 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:02:59.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:02:59.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:02:59.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:02:59.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:02:40.902307+0000 '>' 2026-02-19T13:02:40.902307+0000 2026-02-19T13:02:59.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:03:00.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:03:00.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:03:00.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:03:00.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:03:00.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:03:00.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:03:00.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:03:00.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:02:55.699858+0000 '>' 2026-02-19T13:02:40.902307+0000 2026-02-19T13:03:00.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:03:00.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: seq 1 19 2026-02-19T13:03:00.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:03:00.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ1 2026-02-19T13:03:00.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 1 % 2 2026-02-19T13:03:00.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=1 2026-02-19T13:03:00.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:03:00.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1097: TEST_corrupt_scrub_replicated: local payload=UVWXYZZZ 2026-02-19T13:03:00.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1098: TEST_corrupt_scrub_replicated: echo UVWXYZZZ 2026-02-19T13:03:00.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1099: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:03:00.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:03:00.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:03:00.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:03:00.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:03:00.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:03:00.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:03:00.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:03:00.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:03:00.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:03:00.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:03:00.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:03:00.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:03:00.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:03:00.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:03:00.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:03:00.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:03:00.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:03:00.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:03:00.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:03:00.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:03:00.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:03:00.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:00.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:03:00.935 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#-5:00000000:::deep_scrub_3.0:head#, (61) No data available 2026-02-19T13:03:00.936 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#-5:00000000:::scrub_3.0:head#, (61) No data available 2026-02-19T13:03:01.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:03:01.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:03:01.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:03:01.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:03:01.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:03:01.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:01.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:03:01.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:03:01.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:03:01.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:03:01.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:03:01.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:03:01.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:03:01.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:03:01.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:03:01.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:03:01.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:03:01.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:03:01.025 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:03:01.025 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:03:01.025 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:03:01.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:01.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:01.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:03:01.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:03:01.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:03:01.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:03:01.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:03:01.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:03:01.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:03:01.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:03:01.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:03:01.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:03:01.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:03:01.027 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:03:01.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:03:01.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:03:01.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:03:01.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:03:01.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:03:01.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:03:01.045 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:01.041+0000 7f921bf61a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:01.047 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:01.045+0000 7f921bf61a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:01.048 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:01.045+0000 7f921bf61a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:01.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:03:01.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:03:01.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:03:01.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:03:01.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:03:01.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:01.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:03:01.190 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:03:01.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:01.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:01.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:03:01.372 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:01.369+0000 7f921bf61a40 -1 Falling back to public interface 2026-02-19T13:03:01.942 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:01.937+0000 7f921bf61a40 -1 osd.1 61 log_to_monitors true 2026-02-19T13:03:02.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:03:02.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:02.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:03:02.343 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:03:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:02.499 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 65 up_thru 65 down_at 62 last_clean_interval [10,61) [v2:127.0.0.1:6810/162000539,v1:127.0.0.1:6811/162000539] [v2:127.0.0.1:6812/162000539,v1:127.0.0.1:6813/162000539] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:03:02.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:03:02.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:03:02.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:03:02.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:03:02.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:03:02.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:03:02.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:03:02.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:03:02.500 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:03:02.500 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:03:02.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:03:02.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:03:02.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:03:02.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:03:02.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:03:02.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:03:02.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:03:02.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:03:02.609 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:03:02.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:03:02.761 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:03:02.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:03:02.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:02.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:03:02.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-19T13:03:02.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-19T13:03:02.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491' 2026-02-19T13:03:02.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:02.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:03:02.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=279172874242 2026-02-19T13:03:02.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 279172874242 2026-02-19T13:03:02.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-279172874242' 2026-02-19T13:03:02.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:02.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-19T13:03:02.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:02.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:03:02.908 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836491 2026-02-19T13:03:02.908 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:02.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836491 2026-02-19T13:03:02.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-19T13:03:02.910 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836491 2026-02-19T13:03:02.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:03.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T13:03:03.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:03:04.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:03:04.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:04.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T13:03:04.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:03:05.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:03:05.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:05.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-19T13:03:05.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:05.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-279172874242 2026-02-19T13:03:05.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:05.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:03:05.406 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-279172874242 2026-02-19T13:03:05.406 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:05.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=279172874242 2026-02-19T13:03:05.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 279172874242' 2026-02-19T13:03:05.408 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 279172874242 2026-02-19T13:03:05.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:03:05.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 279172874242 -lt 279172874242 2026-02-19T13:03:05.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:03:05.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:05.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:05.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:03:05.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:03:05.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:03:05.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:03:05.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:03:05.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:03:05.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:03:05.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:03:05.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:03:05.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:03:05.937 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:05.937 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:06.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:03:06.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:03:06.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:03:06.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:03:06.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ2 2026-02-19T13:03:06.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 2 % 2 2026-02-19T13:03:06.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=0 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1104: TEST_corrupt_scrub_replicated: local payload=UVWXYZ 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1105: TEST_corrupt_scrub_replicated: echo UVWXYZ 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1106: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:03:06.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:03:06.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:03:06.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:03:06.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:03:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:03:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:03:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:03:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:03:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:03:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:03:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:03:06.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:03:06.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:03:06.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:03:06.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:03:06.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:03:06.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:03:06.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:03:06.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:03:06.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:03:06.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:03:06.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:03:06.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:03:06.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:03:06.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:03:06.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:03:06.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:03:06.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:03:06.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:03:06.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:03:06.741 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:03:06.741 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:03:06.741 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:03:06.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:06.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:06.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:03:06.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:03:06.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:03:06.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:03:06.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:03:06.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:03:06.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:03:06.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:03:06.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:03:06.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:03:06.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:03:06.743 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:03:06.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:03:06.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:03:06.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:03:06.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:03:06.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:03:06.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:03:06.760 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:06.757+0000 7f78afea5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:06.761 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:06.757+0000 7f78afea5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:06.762 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:06.757+0000 7f78afea5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:06.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:03:06.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:03:06.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:03:06.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:03:06.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:03:06.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:06.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:03:06.901 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:03:06.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:06.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:03:07.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:03:07.156 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:07.153+0000 7f78afea5a40 -1 Falling back to public interface 2026-02-19T13:03:07.787 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:07.785+0000 7f78afea5a40 -1 osd.0 68 log_to_monitors true 2026-02-19T13:03:08.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:03:08.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:08.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:03:08.065 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:03:08.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:08.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:03:08.228 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 71 up_thru 71 down_at 68 last_clean_interval [5,68) [v2:127.0.0.1:6802/2522437983,v1:127.0.0.1:6803/2522437983] [v2:127.0.0.1:6804/2522437983,v1:127.0.0.1:6805/2522437983] exists,up 0250c601-08cf-4db7-8be3-271bfebc9558 2026-02-19T13:03:08.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:03:08.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:03:08.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:03:08.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:03:08.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:03:08.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:03:08.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:03:08.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:03:08.230 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:03:08.230 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:03:08.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:03:08.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:03:08.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:03:08.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:03:08.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:03:08.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:03:08.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:03:08.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:03:08.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:03:08.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:03:08.504 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:03:08.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:03:08.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:08.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:03:08.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=304942678018 2026-02-19T13:03:08.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 304942678018 2026-02-19T13:03:08.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-304942678018' 2026-02-19T13:03:08.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:08.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:03:08.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=279172874244 2026-02-19T13:03:08.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 279172874244 2026-02-19T13:03:08.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-304942678018 1-279172874244' 2026-02-19T13:03:08.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:08.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-304942678018 2026-02-19T13:03:08.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:08.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:03:08.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-304942678018 2026-02-19T13:03:08.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:08.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=304942678018 2026-02-19T13:03:08.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 304942678018' 2026-02-19T13:03:08.661 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 304942678018 2026-02-19T13:03:08.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:08.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 304942678018 2026-02-19T13:03:08.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:03:09.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:03:09.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:09.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 304942678018 2026-02-19T13:03:09.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:03:10.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:03:10.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:11.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 304942678018 -lt 304942678018 2026-02-19T13:03:11.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:11.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-279172874244 2026-02-19T13:03:11.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:11.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:03:11.156 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-279172874244 2026-02-19T13:03:11.156 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:11.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=279172874244 2026-02-19T13:03:11.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 279172874244' 2026-02-19T13:03:11.158 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 279172874244 2026-02-19T13:03:11.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:03:11.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 279172874244 -lt 279172874244 2026-02-19T13:03:11.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:03:11.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:11.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:11.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:03:11.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:03:11.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:03:11.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:03:11.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:03:11.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:03:11.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:03:11.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:03:11.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:03:11.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:03:11.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:11.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:11.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:03:11.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:03:11.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:03:11.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:03:11.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ3 2026-02-19T13:03:11.877 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 3 % 2 2026-02-19T13:03:11.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=1 2026-02-19T13:03:11.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:03:11.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1111: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ3 remove 2026-02-19T13:03:11.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:03:11.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:03:11.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:03:11.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:03:11.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ3 remove 2026-02-19T13:03:11.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:03:11.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:03:11.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:03:11.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:03:11.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:03:11.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:03:11.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:03:11.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:03:11.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:03:11.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:03:12.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:03:12.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ3 remove 2026-02-19T13:03:12.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:03:12.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:03:12.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:03:12.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:03:12.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:12.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ3 remove 2026-02-19T13:03:12.358 INFO:tasks.workunit.client.0.trial144.stdout:remove #3:f2a5b2a4:::ROBJ3:head# 2026-02-19T13:03:12.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:03:12.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:03:12.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:03:12.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:03:12.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:03:12.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:12.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:03:12.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:03:12.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:03:12.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:03:12.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:03:12.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:03:12.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:03:12.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:03:12.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:03:12.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:03:12.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:03:12.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:03:12.470 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:03:12.470 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:03:12.470 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:03:12.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:12.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:12.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:03:12.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:03:12.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:03:12.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:03:12.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:03:12.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:03:12.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:03:12.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:03:12.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:03:12.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:03:12.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:03:12.473 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:03:12.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:03:12.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:03:12.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:03:12.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:03:12.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:03:12.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:03:12.488 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:12.485+0000 7ff218feaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:12.490 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:12.485+0000 7ff218feaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:12.490 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:12.489+0000 7ff218feaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:12.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:03:12.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:03:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:03:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:03:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:03:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:03:12.629 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:03:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:12.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:03:12.900 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:12.897+0000 7ff218feaa40 -1 Falling back to public interface 2026-02-19T13:03:13.486 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:13.481+0000 7ff218feaa40 -1 osd.1 72 log_to_monitors true 2026-02-19T13:03:13.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:03:13.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:13.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:03:13.792 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:03:13.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:13.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:13.951 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 76 up_thru 76 down_at 73 last_clean_interval [65,72) [v2:127.0.0.1:6810/1643255414,v1:127.0.0.1:6811/1643255414] [v2:127.0.0.1:6812/1643255414,v1:127.0.0.1:6813/1643255414] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:03:13.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:03:13.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:03:13.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:03:13.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:03:13.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:03:13.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:03:13.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:03:13.952 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:03:13.953 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:03:13.953 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:03:13.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:03:13.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:03:13.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:03:14.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:03:14.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:03:14.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:03:14.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:03:14.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:03:14.072 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:03:14.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:03:14.230 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:03:14.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:03:14.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:14.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:03:14.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=304942678020 2026-02-19T13:03:14.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 304942678020 2026-02-19T13:03:14.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-304942678020' 2026-02-19T13:03:14.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:14.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:03:14.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=326417514498 2026-02-19T13:03:14.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 326417514498 2026-02-19T13:03:14.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-304942678020 1-326417514498' 2026-02-19T13:03:14.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:14.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-304942678020 2026-02-19T13:03:14.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:14.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:03:14.379 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-304942678020 2026-02-19T13:03:14.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:14.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=304942678020 2026-02-19T13:03:14.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 304942678020' 2026-02-19T13:03:14.381 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 304942678020 2026-02-19T13:03:14.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:14.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 304942678020 -lt 304942678020 2026-02-19T13:03:14.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:14.550 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-326417514498 2026-02-19T13:03:14.550 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:14.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:03:14.552 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-326417514498 2026-02-19T13:03:14.552 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:14.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=326417514498 2026-02-19T13:03:14.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 326417514498' 2026-02-19T13:03:14.554 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 326417514498 2026-02-19T13:03:14.555 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:03:14.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 326417514498 -lt 326417514498 2026-02-19T13:03:14.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:03:14.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:14.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:14.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:03:14.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:03:14.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:03:14.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:03:14.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:03:14.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:03:14.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:03:14.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:03:15.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:03:15.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:03:15.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:15.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:15.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:03:15.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:03:15.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:03:15.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:03:15.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ4 2026-02-19T13:03:15.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 4 % 2 2026-02-19T13:03:15.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=0 2026-02-19T13:03:15.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:03:15.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1116: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-19T13:03:15.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:03:15.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:03:15.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:03:15.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:03:15.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-19T13:03:15.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:03:15.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:03:15.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:03:15.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:03:15.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:03:15.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:03:15.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:03:15.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:03:15.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:03:15.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:03:15.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:03:15.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-19T13:03:15.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:03:15.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:03:15.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:03:15.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:03:15.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:03:15.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-19T13:03:15.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:03:15.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:03:15.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:03:15.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:03:15.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:03:15.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:03:15.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:03:15.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:03:15.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:03:15.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:03:15.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:03:15.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:03:15.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:03:15.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:03:15.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:03:15.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:03:15.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:03:15.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:03:15.866 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:03:15.867 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:03:15.867 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:03:15.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:15.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:15.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:03:15.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:03:15.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:03:15.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:03:15.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:03:15.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:03:15.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:03:15.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:03:15.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:03:15.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:03:15.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:03:15.870 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:03:15.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:03:15.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:03:15.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:03:15.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:03:15.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:03:15.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:03:15.885 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:15.880+0000 7f0fc88d4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:15.887 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:15.884+0000 7f0fc88d4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:15.888 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:15.884+0000 7f0fc88d4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:16.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:03:16.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:03:16.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:03:16.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:03:16.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:03:16.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:16.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:03:16.023 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:03:16.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:16.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:03:16.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:03:16.276 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:16.272+0000 7f0fc88d4a40 -1 Falling back to public interface 2026-02-19T13:03:16.899 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:16.896+0000 7f0fc88d4a40 -1 osd.0 78 log_to_monitors true 2026-02-19T13:03:17.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:03:17.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:17.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:03:17.186 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:03:17.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:17.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:03:17.346 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 82 up_thru 82 down_at 79 last_clean_interval [71,78) [v2:127.0.0.1:6802/81065142,v1:127.0.0.1:6803/81065142] [v2:127.0.0.1:6804/81065142,v1:127.0.0.1:6805/81065142] exists,up 0250c601-08cf-4db7-8be3-271bfebc9558 2026-02-19T13:03:17.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:03:17.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:03:17.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:03:17.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:03:17.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:03:17.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:03:17.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:03:17.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:03:17.348 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:03:17.348 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:03:17.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:03:17.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:03:17.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:03:17.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:03:17.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:03:17.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:03:17.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:03:17.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:03:17.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:03:17.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:03:17.626 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:03:17.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:03:17.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:17.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:03:17.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=352187318274 2026-02-19T13:03:17.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 352187318274 2026-02-19T13:03:17.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-352187318274' 2026-02-19T13:03:17.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:17.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:03:17.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=326417514499 2026-02-19T13:03:17.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 326417514499 2026-02-19T13:03:17.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-352187318274 1-326417514499' 2026-02-19T13:03:17.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:17.771 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-352187318274 2026-02-19T13:03:17.771 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:17.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:03:17.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-352187318274 2026-02-19T13:03:17.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:17.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=352187318274 2026-02-19T13:03:17.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 352187318274' 2026-02-19T13:03:17.775 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 352187318274 2026-02-19T13:03:17.775 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:17.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 304942678020 -lt 352187318274 2026-02-19T13:03:17.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:03:18.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:03:18.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:19.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 352187318274 -lt 352187318274 2026-02-19T13:03:19.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:19.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-326417514499 2026-02-19T13:03:19.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:19.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:03:19.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-326417514499 2026-02-19T13:03:19.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:19.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=326417514499 2026-02-19T13:03:19.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 326417514499' 2026-02-19T13:03:19.099 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 326417514499 2026-02-19T13:03:19.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:03:19.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 326417514499 -lt 326417514499 2026-02-19T13:03:19.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:03:19.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:19.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:19.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:03:19.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:03:19.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:03:19.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:03:19.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:03:19.454 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:03:19.454 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:03:19.454 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:03:19.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:03:19.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:03:19.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:19.615 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:19.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:03:19.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:03:19.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:03:19.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:03:19.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ5 2026-02-19T13:03:19.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 5 % 2 2026-02-19T13:03:19.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=1 2026-02-19T13:03:19.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:03:19.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1121: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ5 rm-omap key-ROBJ5 2026-02-19T13:03:19.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:03:19.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:03:19.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:03:19.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:03:19.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ5 rm-omap key-ROBJ5 2026-02-19T13:03:19.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:03:19.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:03:19.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:03:19.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:03:19.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:03:19.817 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:03:19.817 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:03:19.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:03:19.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:03:19.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:03:20.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:03:20.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ5 rm-omap key-ROBJ5 2026-02-19T13:03:20.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:03:20.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:03:20.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:03:20.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:03:20.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:20.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ5 rm-omap key-ROBJ5 2026-02-19T13:03:20.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:03:20.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:03:20.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:03:20.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:03:20.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:03:20.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:20.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:03:20.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:03:20.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:03:20.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:03:20.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:03:20.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:03:20.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:03:20.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:03:20.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:03:20.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:03:20.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:03:20.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:03:20.426 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:03:20.426 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:03:20.426 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:03:20.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:20.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:20.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:03:20.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:03:20.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:03:20.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:03:20.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:03:20.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:03:20.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:03:20.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:03:20.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:03:20.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:03:20.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:03:20.427 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:03:20.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:03:20.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:03:20.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:03:20.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:03:20.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:03:20.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:03:20.442 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:20.436+0000 7f276834da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:20.444 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:20.440+0000 7f276834da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:20.445 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:20.440+0000 7f276834da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:20.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:03:20.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:03:20.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:03:20.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:03:20.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:03:20.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:20.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:03:20.586 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:03:20.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:20.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:20.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:03:20.828 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:20.824+0000 7f276834da40 -1 Falling back to public interface 2026-02-19T13:03:21.397 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:21.392+0000 7f276834da40 -1 osd.1 84 log_to_monitors true 2026-02-19T13:03:21.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:03:21.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:21.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:03:21.744 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:03:21.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:21.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:21.897 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 88 up_thru 88 down_at 85 last_clean_interval [76,84) [v2:127.0.0.1:6810/3472374355,v1:127.0.0.1:6811/3472374355] [v2:127.0.0.1:6812/3472374355,v1:127.0.0.1:6813/3472374355] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:03:21.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:03:21.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:03:21.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:03:21.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:03:21.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:03:21.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:03:21.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:03:21.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:03:21.898 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:03:21.898 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:03:21.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:03:21.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:03:21.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:03:22.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:03:22.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:03:22.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:03:22.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:03:22.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:03:22.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:03:22.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:03:22.178 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:03:22.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:03:22.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:22.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:03:22.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=352187318276 2026-02-19T13:03:22.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 352187318276 2026-02-19T13:03:22.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-352187318276' 2026-02-19T13:03:22.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:22.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:03:22.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=377957122050 2026-02-19T13:03:22.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 377957122050 2026-02-19T13:03:22.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-352187318276 1-377957122050' 2026-02-19T13:03:22.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:22.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-352187318276 2026-02-19T13:03:22.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:22.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:03:22.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-352187318276 2026-02-19T13:03:22.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:22.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=352187318276 2026-02-19T13:03:22.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 352187318276' 2026-02-19T13:03:22.326 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 352187318276 2026-02-19T13:03:22.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:22.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 352187318276 -lt 352187318276 2026-02-19T13:03:22.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:22.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-377957122050 2026-02-19T13:03:22.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:22.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:03:22.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-377957122050 2026-02-19T13:03:22.494 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:22.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=377957122050 2026-02-19T13:03:22.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 377957122050' 2026-02-19T13:03:22.495 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 377957122050 2026-02-19T13:03:22.496 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:03:22.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 377957122050 -lt 377957122050 2026-02-19T13:03:22.654 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:03:22.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:22.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:22.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:03:22.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:03:22.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:03:22.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:03:22.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:03:22.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:03:22.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:03:22.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:03:23.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:03:23.003 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:03:23.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:23.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:23.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:03:23.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:03:23.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:03:23.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:03:23.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ6 2026-02-19T13:03:23.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 6 % 2 2026-02-19T13:03:23.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=0 2026-02-19T13:03:23.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:03:23.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1126: TEST_corrupt_scrub_replicated: echo extra 2026-02-19T13:03:23.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1127: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-19T13:03:23.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:03:23.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:03:23.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:03:23.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:03:23.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-19T13:03:23.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:03:23.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:03:23.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:03:23.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:03:23.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:03:23.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:03:23.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:03:23.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:03:23.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:03:23.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:03:23.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:03:23.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-19T13:03:23.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:03:23.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:03:23.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:03:23.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:03:23.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:03:23.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-19T13:03:23.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:03:23.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:03:23.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:03:23.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:03:23.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:03:23.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:03:23.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:03:23.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:03:23.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:03:23.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:03:23.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:03:23.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:03:23.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:03:23.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:03:23.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:03:23.807 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:03:23.807 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:03:23.807 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:03:23.807 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:03:23.807 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:03:23.807 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:03:23.807 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:23.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:23.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:03:23.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:03:23.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:03:23.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:03:23.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:03:23.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:03:23.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:03:23.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:03:23.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:03:23.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:03:23.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:03:23.810 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:03:23.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:03:23.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:03:23.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:03:23.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:03:23.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:03:23.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:03:23.826 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:23.820+0000 7feb61017a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:23.827 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:23.824+0000 7feb61017a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:23.829 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:23.824+0000 7feb61017a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:23.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:03:23.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:03:23.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:03:23.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:03:23.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:03:23.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:23.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:03:23.970 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:03:23.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:23.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:03:24.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:03:24.268 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:24.264+0000 7feb61017a40 -1 Falling back to public interface 2026-02-19T13:03:24.845 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:24.840+0000 7feb61017a40 -1 osd.0 89 log_to_monitors true 2026-02-19T13:03:25.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:03:25.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:25.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:03:25.131 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:03:25.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:25.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:03:25.289 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 93 up_thru 93 down_at 90 last_clean_interval [82,89) [v2:127.0.0.1:6802/1189742896,v1:127.0.0.1:6803/1189742896] [v2:127.0.0.1:6804/1189742896,v1:127.0.0.1:6805/1189742896] exists,up 0250c601-08cf-4db7-8be3-271bfebc9558 2026-02-19T13:03:25.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:03:25.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:03:25.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:03:25.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:03:25.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:03:25.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:03:25.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:03:25.290 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:03:25.291 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:03:25.291 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:03:25.291 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:03:25.291 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:03:25.291 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:03:25.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:03:25.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:03:25.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:03:25.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:03:25.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:03:25.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:03:25.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:03:25.564 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:03:25.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:03:25.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:25.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:03:25.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=399431958530 2026-02-19T13:03:25.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 399431958530 2026-02-19T13:03:25.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-399431958530' 2026-02-19T13:03:25.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:25.637 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:03:25.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=377957122051 2026-02-19T13:03:25.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 377957122051 2026-02-19T13:03:25.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-399431958530 1-377957122051' 2026-02-19T13:03:25.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:25.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-399431958530 2026-02-19T13:03:25.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:25.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:03:25.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-399431958530 2026-02-19T13:03:25.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:25.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=399431958530 2026-02-19T13:03:25.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 399431958530' 2026-02-19T13:03:25.713 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 399431958530 2026-02-19T13:03:25.713 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:25.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 352187318276 -lt 399431958530 2026-02-19T13:03:25.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:03:26.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:03:26.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:27.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 399431958530 -lt 399431958530 2026-02-19T13:03:27.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:27.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-377957122051 2026-02-19T13:03:27.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:27.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:03:27.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-377957122051 2026-02-19T13:03:27.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:27.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=377957122051 2026-02-19T13:03:27.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 377957122051' 2026-02-19T13:03:27.049 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 377957122051 2026-02-19T13:03:27.049 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:03:27.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 377957122051 -lt 377957122051 2026-02-19T13:03:27.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:03:27.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:27.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:27.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:03:27.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:03:27.397 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:03:27.397 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:03:27.397 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:03:27.397 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:03:27.397 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:03:27.397 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:03:27.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:03:27.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:03:27.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:27.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:27.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:03:27.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:03:27.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:03:27.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1128: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/extra-val 2026-02-19T13:03:27.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:03:27.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ7 2026-02-19T13:03:27.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 7 % 2 2026-02-19T13:03:27.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=1 2026-02-19T13:03:27.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:03:27.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1133: TEST_corrupt_scrub_replicated: echo -n newheader 2026-02-19T13:03:27.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1134: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-19T13:03:27.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:03:27.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:03:27.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:03:27.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:03:27.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-19T13:03:27.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:03:27.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:03:27.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:03:27.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:03:27.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:03:27.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:03:27.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:03:27.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:03:27.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:03:27.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:03:28.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:03:28.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-19T13:03:28.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:03:28.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:03:28.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:03:28.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:03:28.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:28.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-19T13:03:28.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:03:28.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:03:28.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:03:28.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:03:28.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:03:28.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:28.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:03:28.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:03:28.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:03:28.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:03:28.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:03:28.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:03:28.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:03:28.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:03:28.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:03:28.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:03:28.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:03:28.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:03:28.310 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:03:28.310 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:03:28.310 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:03:28.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:28.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:28.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:03:28.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:03:28.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:03:28.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:03:28.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:03:28.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:03:28.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:03:28.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:03:28.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:03:28.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:03:28.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:03:28.313 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:03:28.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:03:28.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:03:28.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:03:28.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:03:28.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:03:28.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:03:28.330 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:28.324+0000 7f8d77e2ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:28.331 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:28.328+0000 7f8d77e2ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:28.332 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:28.328+0000 7f8d77e2ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:28.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:03:28.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:03:28.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:03:28.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:03:28.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:03:28.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:28.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:03:28.476 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:03:28.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:28.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:28.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:03:28.736 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:28.732+0000 7f8d77e2ea40 -1 Falling back to public interface 2026-02-19T13:03:29.308 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:29.304+0000 7f8d77e2ea40 -1 osd.1 95 log_to_monitors true 2026-02-19T13:03:29.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:03:29.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:29.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:03:29.633 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:03:29.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:29.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:29.796 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 99 up_thru 99 down_at 96 last_clean_interval [88,95) [v2:127.0.0.1:6810/1378903812,v1:127.0.0.1:6811/1378903812] [v2:127.0.0.1:6812/1378903812,v1:127.0.0.1:6813/1378903812] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:03:29.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:03:29.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:03:29.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:03:29.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:03:29.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:03:29.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:03:29.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:03:29.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:03:29.798 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:03:29.798 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:03:29.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:03:29.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:03:29.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:03:29.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:03:29.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:03:29.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:03:29.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:03:29.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:03:29.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:03:30.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:03:30.076 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:03:30.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:03:30.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:30.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:03:30.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=399431958532 2026-02-19T13:03:30.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 399431958532 2026-02-19T13:03:30.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-399431958532' 2026-02-19T13:03:30.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:30.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:03:30.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=425201762306 2026-02-19T13:03:30.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 425201762306 2026-02-19T13:03:30.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-399431958532 1-425201762306' 2026-02-19T13:03:30.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:30.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-399431958532 2026-02-19T13:03:30.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:30.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:03:30.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-399431958532 2026-02-19T13:03:30.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:30.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=399431958532 2026-02-19T13:03:30.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 399431958532' 2026-02-19T13:03:30.226 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 399431958532 2026-02-19T13:03:30.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:30.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 399431958530 -lt 399431958532 2026-02-19T13:03:30.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:03:31.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:03:31.387 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:31.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 399431958532 -lt 399431958532 2026-02-19T13:03:31.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:31.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-425201762306 2026-02-19T13:03:31.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:31.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:03:31.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-425201762306 2026-02-19T13:03:31.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:31.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=425201762306 2026-02-19T13:03:31.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 425201762306' 2026-02-19T13:03:31.551 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 425201762306 2026-02-19T13:03:31.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:03:31.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 425201762306 -lt 425201762306 2026-02-19T13:03:31.716 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:03:31.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:31.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:31.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:03:31.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:03:31.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:03:31.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:03:31.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:03:31.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:03:31.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:03:31.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:03:32.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:03:32.072 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:03:32.072 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:32.072 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:32.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:03:32.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:03:32.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:03:32.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1135: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/hdr 2026-02-19T13:03:32.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:03:32.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ8 2026-02-19T13:03:32.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 8 % 2 2026-02-19T13:03:32.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=0 2026-02-19T13:03:32.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:03:32.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1139: TEST_corrupt_scrub_replicated: rados --pool csr_pool setxattr ROBJ8 key1-ROBJ8 val1-ROBJ8 2026-02-19T13:03:32.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1140: TEST_corrupt_scrub_replicated: rados --pool csr_pool setxattr ROBJ8 key2-ROBJ8 val2-ROBJ8 2026-02-19T13:03:32.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1143: TEST_corrupt_scrub_replicated: echo -n bad-val 2026-02-19T13:03:32.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1144: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-19T13:03:32.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:03:32.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:03:32.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:03:32.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:03:32.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-19T13:03:32.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:03:32.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:03:32.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:03:32.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:03:32.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:03:32.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:03:32.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:03:32.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:03:32.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:03:32.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:03:32.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:03:32.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-19T13:03:32.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:03:32.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:03:32.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:03:32.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:03:32.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:03:32.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-19T13:03:32.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:03:32.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:03:32.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:03:32.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:03:32.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:03:32.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:03:32.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:03:32.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:03:32.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:03:32.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:03:32.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:03:32.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:03:32.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:03:32.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:03:32.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:03:32.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:03:32.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:03:32.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:03:32.906 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:03:32.906 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:03:32.907 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:03:32.907 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:32.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:32.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:03:32.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:03:32.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:03:32.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:03:32.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:03:32.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:03:32.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:03:32.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:03:32.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:03:32.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:03:32.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:03:32.908 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:03:32.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:03:32.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:03:32.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:03:32.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:03:32.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:03:32.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:03:32.926 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:32.920+0000 7f1930b85a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:32.927 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:32.924+0000 7f1930b85a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:32.928 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:32.924+0000 7f1930b85a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:33.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:03:33.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:03:33.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:03:33.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:03:33.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:03:33.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:33.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:03:33.070 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:03:33.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:33.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:03:33.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:03:33.292 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:33.288+0000 7f1930b85a40 -1 Falling back to public interface 2026-02-19T13:03:33.831 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:33.828+0000 7f1930b85a40 -1 osd.0 101 log_to_monitors true 2026-02-19T13:03:34.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:03:34.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:34.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:03:34.233 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:03:34.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:34.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:03:34.391 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 105 up_thru 105 down_at 102 last_clean_interval [93,101) [v2:127.0.0.1:6802/3945694672,v1:127.0.0.1:6803/3945694672] [v2:127.0.0.1:6804/3945694672,v1:127.0.0.1:6805/3945694672] exists,up 0250c601-08cf-4db7-8be3-271bfebc9558 2026-02-19T13:03:34.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:03:34.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:03:34.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:03:34.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:03:34.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:03:34.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:03:34.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:03:34.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:03:34.392 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:03:34.392 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:03:34.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:03:34.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:03:34.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:03:34.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:03:34.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:03:34.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:03:34.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:03:34.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:03:34.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:03:34.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:03:34.665 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:03:34.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:03:34.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:34.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:03:34.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=450971566082 2026-02-19T13:03:34.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 450971566082 2026-02-19T13:03:34.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-450971566082' 2026-02-19T13:03:34.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:34.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:03:34.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=425201762308 2026-02-19T13:03:34.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 425201762308 2026-02-19T13:03:34.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-450971566082 1-425201762308' 2026-02-19T13:03:34.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:34.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-450971566082 2026-02-19T13:03:34.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:34.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:03:34.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-450971566082 2026-02-19T13:03:34.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:34.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=450971566082 2026-02-19T13:03:34.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 450971566082' 2026-02-19T13:03:34.817 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 450971566082 2026-02-19T13:03:34.817 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:34.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 450971566082 2026-02-19T13:03:34.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:03:35.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:03:35.985 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:36.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 450971566082 2026-02-19T13:03:36.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:03:37.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:03:37.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:37.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 450971566082 -lt 450971566082 2026-02-19T13:03:37.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:37.308 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-425201762308 2026-02-19T13:03:37.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:37.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:03:37.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-425201762308 2026-02-19T13:03:37.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:37.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=425201762308 2026-02-19T13:03:37.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 425201762308' 2026-02-19T13:03:37.313 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 425201762308 2026-02-19T13:03:37.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:03:37.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 425201762308 -lt 425201762308 2026-02-19T13:03:37.477 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:03:37.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:37.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:37.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:03:37.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:03:37.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:03:37.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:03:37.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:03:37.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:03:37.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:03:37.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:03:37.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:03:37.827 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:03:37.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:37.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1145: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:03:38.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:03:38.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:03:38.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:03:38.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:03:38.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:03:38.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-19T13:03:38.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:03:38.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:03:38.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:03:38.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:03:38.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:38.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-19T13:03:38.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:03:38.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:03:38.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:03:38.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:03:38.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:03:38.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:38.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:03:38.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:03:38.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:03:38.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:03:38.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:03:38.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:03:38.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:03:38.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:03:38.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:03:38.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:03:38.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:03:38.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:03:38.630 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:03:38.630 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:03:38.630 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:03:38.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:38.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:38.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:03:38.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:03:38.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:03:38.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:03:38.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:03:38.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:03:38.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:03:38.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:03:38.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:03:38.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:03:38.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:03:38.633 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:03:38.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:03:38.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:03:38.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:03:38.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:03:38.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:03:38.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:03:38.650 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:38.644+0000 7ff6e8453a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:38.651 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:38.648+0000 7ff6e8453a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:38.652 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:38.648+0000 7ff6e8453a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:38.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:03:38.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:03:38.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:03:38.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:03:38.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:03:38.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:38.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:03:38.793 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:03:38.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:38.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:38.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:03:39.048 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:39.044+0000 7ff6e8453a40 -1 Falling back to public interface 2026-02-19T13:03:39.667 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:39.664+0000 7ff6e8453a40 -1 osd.1 106 log_to_monitors true 2026-02-19T13:03:39.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:03:39.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:39.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:03:39.957 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:03:39.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:39.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:40.110 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 110 up_thru 110 down_at 107 last_clean_interval [99,106) [v2:127.0.0.1:6810/2437017004,v1:127.0.0.1:6811/2437017004] [v2:127.0.0.1:6812/2437017004,v1:127.0.0.1:6813/2437017004] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:03:40.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:03:40.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:03:40.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:03:40.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:03:40.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:03:40.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:03:40.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:03:40.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:03:40.112 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:03:40.112 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:03:40.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:03:40.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:03:40.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:03:40.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:03:40.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:03:40.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:03:40.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:03:40.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:03:40.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:03:40.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:03:40.386 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:03:40.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:03:40.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:40.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:03:40.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=450971566084 2026-02-19T13:03:40.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 450971566084 2026-02-19T13:03:40.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-450971566084' 2026-02-19T13:03:40.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:40.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:03:40.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=472446402562 2026-02-19T13:03:40.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 472446402562 2026-02-19T13:03:40.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-450971566084 1-472446402562' 2026-02-19T13:03:40.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:40.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-450971566084 2026-02-19T13:03:40.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:40.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:03:40.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-450971566084 2026-02-19T13:03:40.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:40.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=450971566084 2026-02-19T13:03:40.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 450971566084' 2026-02-19T13:03:40.550 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 450971566084 2026-02-19T13:03:40.550 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:40.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 450971566083 -lt 450971566084 2026-02-19T13:03:40.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:03:41.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:03:41.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:41.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 450971566083 -lt 450971566084 2026-02-19T13:03:41.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:03:42.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:03:42.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:43.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 450971566084 -lt 450971566084 2026-02-19T13:03:43.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:43.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-472446402562 2026-02-19T13:03:43.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:43.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:03:43.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-472446402562 2026-02-19T13:03:43.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:43.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=472446402562 2026-02-19T13:03:43.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 472446402562' 2026-02-19T13:03:43.040 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 472446402562 2026-02-19T13:03:43.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:03:43.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 472446402562 -lt 472446402562 2026-02-19T13:03:43.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:03:43.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:43.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:43.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:03:43.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:03:43.388 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:03:43.388 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:03:43.388 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:03:43.388 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:03:43.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:03:43.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:03:43.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:03:43.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:03:43.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:43.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:43.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:03:43.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:03:43.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:03:43.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1146: TEST_corrupt_scrub_replicated: echo -n val3-ROBJ8 2026-02-19T13:03:43.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1147: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-19T13:03:43.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:03:43.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:03:43.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:03:43.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:03:43.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-19T13:03:43.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:03:43.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:03:43.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:03:43.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:03:43.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:03:43.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:03:43.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:03:43.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:03:43.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:03:43.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:03:44.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:03:44.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-19T13:03:44.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:03:44.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:03:44.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:03:44.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:03:44.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:44.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-19T13:03:44.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:03:44.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:03:44.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:03:44.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:03:44.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:03:44.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:44.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:03:44.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:03:44.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:03:44.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:03:44.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:03:44.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:03:44.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:03:44.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:03:44.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:03:44.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:03:44.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:03:44.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:03:44.306 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:03:44.306 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:03:44.306 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:03:44.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:44.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:44.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:03:44.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:03:44.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:03:44.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:03:44.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:03:44.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:03:44.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:03:44.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:03:44.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:03:44.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:03:44.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:03:44.307 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:03:44.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:03:44.308 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:03:44.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:03:44.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:03:44.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:03:44.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:03:44.327 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:44.328+0000 7f4c22ae4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:44.328 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:44.328+0000 7f4c22ae4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:44.329 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:44.328+0000 7f4c22ae4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:44.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:03:44.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:03:44.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:03:44.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:03:44.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:03:44.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:44.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:03:44.466 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:03:44.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:44.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:44.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:03:44.668 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:44.668+0000 7f4c22ae4a40 -1 Falling back to public interface 2026-02-19T13:03:45.274 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:45.272+0000 7f4c22ae4a40 -1 osd.1 111 log_to_monitors true 2026-02-19T13:03:45.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:03:45.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:45.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:03:45.626 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:03:45.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:45.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:45.787 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 115 up_thru 115 down_at 112 last_clean_interval [110,111) [v2:127.0.0.1:6810/3215085644,v1:127.0.0.1:6811/3215085644] [v2:127.0.0.1:6812/3215085644,v1:127.0.0.1:6813/3215085644] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:03:45.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:03:45.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:03:45.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:03:45.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:03:45.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:03:45.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:03:45.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:03:45.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:03:45.789 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:03:45.789 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:03:45.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:03:45.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:03:45.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:03:45.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:03:45.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:03:45.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:03:45.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:03:45.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:03:45.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:03:46.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:03:46.068 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:03:46.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:03:46.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:46.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:03:46.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=450971566086 2026-02-19T13:03:46.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 450971566086 2026-02-19T13:03:46.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-450971566086' 2026-02-19T13:03:46.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:46.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:03:46.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=493921239042 2026-02-19T13:03:46.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 493921239042 2026-02-19T13:03:46.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-450971566086 1-493921239042' 2026-02-19T13:03:46.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:46.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-450971566086 2026-02-19T13:03:46.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:46.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:03:46.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-450971566086 2026-02-19T13:03:46.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:46.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=450971566086 2026-02-19T13:03:46.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 450971566086' 2026-02-19T13:03:46.222 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 450971566086 2026-02-19T13:03:46.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:46.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 450971566085 -lt 450971566086 2026-02-19T13:03:46.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:03:47.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:03:47.379 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:47.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 450971566086 -lt 450971566086 2026-02-19T13:03:47.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:47.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-493921239042 2026-02-19T13:03:47.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:47.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:03:47.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-493921239042 2026-02-19T13:03:47.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:47.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=493921239042 2026-02-19T13:03:47.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 493921239042' 2026-02-19T13:03:47.548 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 493921239042 2026-02-19T13:03:47.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:03:47.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 493921239042 -lt 493921239042 2026-02-19T13:03:47.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:03:47.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:47.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:47.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:03:47.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:03:47.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:03:47.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:03:47.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:03:47.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:03:47.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:03:47.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:03:48.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:03:48.055 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:03:48.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:48.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:48.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:03:48.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:03:48.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:03:48.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1148: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/bad-val td/osd-scrub-repair/newval 2026-02-19T13:03:48.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:03:48.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ9 2026-02-19T13:03:48.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 9 % 2 2026-02-19T13:03:48.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=1 2026-02-19T13:03:48.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:03:48.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1152: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ9 get-attr _ 2026-02-19T13:03:48.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:03:48.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:03:48.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:03:48.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:03:48.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ9 get-attr _ 2026-02-19T13:03:48.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:03:48.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:03:48.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:03:48.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:03:48.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:03:48.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:03:48.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:03:48.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:03:48.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:03:48.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:03:48.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:03:48.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ9 get-attr _ 2026-02-19T13:03:48.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:03:48.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:03:48.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:03:48.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:03:48.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:48.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ9 get-attr _ 2026-02-19T13:03:48.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:03:48.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:03:48.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:03:48.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:03:48.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:03:48.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:48.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:03:48.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:03:48.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:03:48.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:03:48.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:03:48.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:03:48.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:03:48.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:03:48.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:03:48.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:03:48.729 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:03:48.729 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:03:48.729 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:03:48.729 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:03:48.729 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:03:48.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:48.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:48.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:03:48.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:03:48.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:03:48.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:03:48.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:03:48.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:03:48.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:03:48.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:03:48.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:03:48.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:03:48.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:03:48.731 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:03:48.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:03:48.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:03:48.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:03:48.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:03:48.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:03:48.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:03:48.747 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:48.748+0000 7f194e2c9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:48.749 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:48.748+0000 7f194e2c9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:48.749 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:48.748+0000 7f194e2c9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:48.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:03:48.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:03:48.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:03:48.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:03:48.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:03:48.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:48.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:03:48.891 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:03:48.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:48.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:49.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:03:49.184 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:49.184+0000 7f194e2c9a40 -1 Falling back to public interface 2026-02-19T13:03:49.784 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:49.784+0000 7f194e2c9a40 -1 osd.1 116 log_to_monitors true 2026-02-19T13:03:50.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:03:50.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:50.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:03:50.054 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:03:50.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:50.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:50.212 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 120 up_thru 120 down_at 117 last_clean_interval [115,116) [v2:127.0.0.1:6810/965691270,v1:127.0.0.1:6811/965691270] [v2:127.0.0.1:6812/965691270,v1:127.0.0.1:6813/965691270] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:03:50.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:03:50.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:03:50.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:03:50.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:03:50.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:03:50.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:03:50.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:03:50.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:03:50.214 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:03:50.214 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:03:50.214 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:03:50.214 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:03:50.214 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:03:50.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:03:50.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:03:50.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:03:50.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:03:50.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:03:50.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:03:50.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:03:50.477 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:03:50.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:03:50.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:50.477 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:03:50.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=450971566088 2026-02-19T13:03:50.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 450971566088 2026-02-19T13:03:50.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-450971566088' 2026-02-19T13:03:50.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:50.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:03:50.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=515396075522 2026-02-19T13:03:50.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 515396075522 2026-02-19T13:03:50.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-450971566088 1-515396075522' 2026-02-19T13:03:50.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:50.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-450971566088 2026-02-19T13:03:50.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:50.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:03:50.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-450971566088 2026-02-19T13:03:50.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:50.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=450971566088 2026-02-19T13:03:50.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 450971566088' 2026-02-19T13:03:50.627 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 450971566088 2026-02-19T13:03:50.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:50.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 450971566087 -lt 450971566088 2026-02-19T13:03:50.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:03:51.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:03:51.787 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:51.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 450971566087 -lt 450971566088 2026-02-19T13:03:51.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:03:52.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:03:52.953 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:53.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 450971566088 -lt 450971566088 2026-02-19T13:03:53.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:53.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-515396075522 2026-02-19T13:03:53.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:53.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:03:53.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-515396075522 2026-02-19T13:03:53.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:53.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=515396075522 2026-02-19T13:03:53.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 515396075522' 2026-02-19T13:03:53.114 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 515396075522 2026-02-19T13:03:53.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:03:53.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 515396075522 -lt 515396075522 2026-02-19T13:03:53.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:03:53.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:53.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:53.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:03:53.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:03:53.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:03:53.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:03:53.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:03:53.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:03:53.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:03:53.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:03:53.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:03:53.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:03:53.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:53.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:53.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:03:53.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:03:53.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:03:53.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1153: TEST_corrupt_scrub_replicated: echo -n D 2026-02-19T13:03:53.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1154: TEST_corrupt_scrub_replicated: rados --pool csr_pool put ROBJ9 td/osd-scrub-repair/change 2026-02-19T13:03:53.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1155: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-19T13:03:53.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:03:53.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:03:53.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:03:53.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:03:53.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-19T13:03:53.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:03:53.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:03:53.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:03:53.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:03:53.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:03:53.841 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:03:53.841 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:03:53.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:03:53.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:03:53.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:03:54.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:03:54.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-19T13:03:54.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:03:54.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:03:54.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:03:54.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:03:54.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:54.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-19T13:03:54.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:03:54.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:03:54.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:03:54.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:03:54.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:03:54.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:03:54.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:03:54.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:03:54.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:03:54.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:03:54.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:03:54.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:03:54.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:03:54.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:03:54.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:03:54.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:03:54.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:03:54.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:03:54.409 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:03:54.409 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:03:54.410 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:03:54.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:54.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:54.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:03:54.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:03:54.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:03:54.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:03:54.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:03:54.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:03:54.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:03:54.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:03:54.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:03:54.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:03:54.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:03:54.411 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:03:54.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:03:54.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:03:54.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:03:54.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:03:54.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:03:54.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:03:54.430 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:54.428+0000 7fcdf5d9ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:54.431 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:54.432+0000 7fcdf5d9ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:54.432 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:54.432+0000 7fcdf5d9ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:54.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:03:54.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:03:54.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:03:54.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:03:54.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:03:54.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:54.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:03:54.574 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:03:54.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:54.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:54.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:03:54.836 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:54.836+0000 7fcdf5d9ba40 -1 Falling back to public interface 2026-02-19T13:03:55.435 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:55.436+0000 7fcdf5d9ba40 -1 osd.1 121 log_to_monitors true 2026-02-19T13:03:55.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:03:55.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:55.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:03:55.729 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:03:55.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:55.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:03:55.883 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 125 up_thru 125 down_at 122 last_clean_interval [120,121) [v2:127.0.0.1:6810/3985092035,v1:127.0.0.1:6811/3985092035] [v2:127.0.0.1:6812/3985092035,v1:127.0.0.1:6813/3985092035] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:03:55.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:03:55.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:03:55.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:03:55.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:03:55.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:03:55.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:03:55.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:03:55.884 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:03:55.885 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:03:55.886 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:03:55.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:03:55.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:03:55.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:03:56.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:03:56.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:03:56.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:03:56.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:03:56.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:03:56.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:03:56.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:03:56.163 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:03:56.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:03:56.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:56.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:03:56.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=450971566090 2026-02-19T13:03:56.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 450971566090 2026-02-19T13:03:56.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-450971566090' 2026-02-19T13:03:56.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:56.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:03:56.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=536870912002 2026-02-19T13:03:56.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 536870912002 2026-02-19T13:03:56.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-450971566090 1-536870912002' 2026-02-19T13:03:56.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:56.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-450971566090 2026-02-19T13:03:56.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:56.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:03:56.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-450971566090 2026-02-19T13:03:56.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:56.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=450971566090 2026-02-19T13:03:56.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 450971566090' 2026-02-19T13:03:56.318 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 450971566090 2026-02-19T13:03:56.318 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:56.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 450971566090 -lt 450971566090 2026-02-19T13:03:56.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:56.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-536870912002 2026-02-19T13:03:56.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:56.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:03:56.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-536870912002 2026-02-19T13:03:56.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:56.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=536870912002 2026-02-19T13:03:56.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 536870912002' 2026-02-19T13:03:56.491 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 536870912002 2026-02-19T13:03:56.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:03:56.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 536870912002 -lt 536870912002 2026-02-19T13:03:56.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:03:56.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:56.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:56.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:03:56.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:03:56.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:03:56.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:03:56.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:03:56.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:03:56.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:03:56.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:03:56.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:03:56.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:03:56.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:03:56.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:03:57.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:03:57.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:03:57.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:03:57.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1156: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/oi td/osd-scrub-repair/change 2026-02-19T13:03:57.201 INFO:tasks.workunit.client.0.trial144.stderr:rm: cannot remove 'td/osd-scrub-repair/oi': No such file or directory 2026-02-19T13:03:57.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:03:57.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ10 2026-02-19T13:03:57.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 10 % 2 2026-02-19T13:03:57.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=0 2026-02-19T13:03:57.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:03:57.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:03:57.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ11 2026-02-19T13:03:57.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 11 % 2 2026-02-19T13:03:57.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=1 2026-02-19T13:03:57.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:03:57.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:03:57.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ12 2026-02-19T13:03:57.205 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 12 % 2 2026-02-19T13:03:57.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=0 2026-02-19T13:03:57.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:03:57.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:03:57.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ13 2026-02-19T13:03:57.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 13 % 2 2026-02-19T13:03:57.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=1 2026-02-19T13:03:57.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:03:57.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:03:57.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ14 2026-02-19T13:03:57.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 14 % 2 2026-02-19T13:03:57.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=0 2026-02-19T13:03:57.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:03:57.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1165: TEST_corrupt_scrub_replicated: echo -n bad-val 2026-02-19T13:03:57.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1166: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-19T13:03:57.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:03:57.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:03:57.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:03:57.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:03:57.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-19T13:03:57.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:03:57.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:03:57.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:03:57.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:03:57.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:03:57.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:03:57.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:03:57.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:03:57.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:03:57.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:03:57.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:03:57.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-19T13:03:57.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:03:57.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:03:57.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:03:57.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:03:57.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:03:57.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-19T13:03:57.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:03:57.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:03:57.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:03:57.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:03:57.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:03:57.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:03:57.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:03:57.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:03:57.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:03:57.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:03:57.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:03:57.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:03:57.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:03:57.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:03:57.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:03:57.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:03:57.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:03:57.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:03:57.810 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:03:57.810 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:03:57.810 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:03:57.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:57.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:03:57.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:03:57.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:03:57.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:03:57.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:03:57.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:03:57.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:03:57.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:03:57.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:03:57.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:03:57.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:03:57.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:03:57.813 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:03:57.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:03:57.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:03:57.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:03:57.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:03:57.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:03:57.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:03:57.830 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:57.828+0000 7f41271a6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:57.831 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:57.832+0000 7f41271a6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:57.832 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:57.832+0000 7f41271a6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:03:57.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:03:57.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:03:57.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:03:57.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:03:57.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:03:57.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:57.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:03:57.972 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:03:57.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:57.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:03:58.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:03:58.240 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:58.240+0000 7f41271a6a40 -1 Falling back to public interface 2026-02-19T13:03:58.831 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:03:58.832+0000 7f41271a6a40 -1 osd.0 126 log_to_monitors true 2026-02-19T13:03:59.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:03:59.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:03:59.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:03:59.135 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:03:59.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:03:59.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:03:59.292 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 130 up_thru 130 down_at 127 last_clean_interval [105,126) [v2:127.0.0.1:6802/273383882,v1:127.0.0.1:6803/273383882] [v2:127.0.0.1:6804/273383882,v1:127.0.0.1:6805/273383882] exists,up 0250c601-08cf-4db7-8be3-271bfebc9558 2026-02-19T13:03:59.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:03:59.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:03:59.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:03:59.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:03:59.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:03:59.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:03:59.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:03:59.292 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:03:59.293 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:03:59.293 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:03:59.293 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:03:59.293 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:03:59.293 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:03:59.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:03:59.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:03:59.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:03:59.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:03:59.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:03:59.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:03:59.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:03:59.570 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:03:59.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:03:59.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:59.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:03:59.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=558345748482 2026-02-19T13:03:59.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 558345748482 2026-02-19T13:03:59.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-558345748482' 2026-02-19T13:03:59.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:03:59.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:03:59.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=536870912003 2026-02-19T13:03:59.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 536870912003 2026-02-19T13:03:59.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-558345748482 1-536870912003' 2026-02-19T13:03:59.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:03:59.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-558345748482 2026-02-19T13:03:59.716 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:03:59.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:03:59.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-558345748482 2026-02-19T13:03:59.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:03:59.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=558345748482 2026-02-19T13:03:59.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 558345748482' 2026-02-19T13:03:59.719 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 558345748482 2026-02-19T13:03:59.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:03:59.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 450971566090 -lt 558345748482 2026-02-19T13:03:59.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:04:00.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:04:00.883 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:01.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 558345748482 -lt 558345748482 2026-02-19T13:04:01.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:01.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-536870912003 2026-02-19T13:04:01.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:01.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:04:01.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-536870912003 2026-02-19T13:04:01.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:01.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=536870912003 2026-02-19T13:04:01.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 536870912003' 2026-02-19T13:04:01.049 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 536870912003 2026-02-19T13:04:01.049 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:04:01.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 536870912003 -lt 536870912003 2026-02-19T13:04:01.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:04:01.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:01.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:01.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:04:01.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:04:01.407 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:04:01.407 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:04:01.407 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:04:01.407 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:04:01.407 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:04:01.407 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:04:01.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:04:01.566 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:04:01.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:01.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:01.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:04:01.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:04:01.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:04:01.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1167: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ14 rm-attr _ 2026-02-19T13:04:01.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:04:01.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:04:01.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:04:01.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:04:01.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ14 rm-attr _ 2026-02-19T13:04:01.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:04:01.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:04:01.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:04:01.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:04:01.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:04:01.766 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:04:01.766 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:04:01.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:04:01.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:04:01.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:04:02.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:04:02.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ14 rm-attr _ 2026-02-19T13:04:02.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:04:02.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:04:02.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:04:02.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:04:02.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:04:02.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ14 rm-attr _ 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:04:02.343 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:04:02.344 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:04:02.344 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:02.344 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:02.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:02.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:02.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:04:02.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:04:02.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:04:02.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:04:02.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:04:02.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:04:02.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:04:02.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:04:02.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:04:02.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:04:02.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:04:02.346 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:04:02.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:04:02.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:04:02.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:04:02.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:04:02.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:04:02.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:04:02.363 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:02.364+0000 7f91fde1ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:02.364 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:02.364+0000 7f91fde1ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:02.365 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:02.364+0000 7f91fde1ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:02.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:04:02.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:04:02.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:04:02.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:04:02.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:04:02.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:02.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:04:02.503 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:04:02.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:02.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:04:02.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:04:02.792 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:02.792+0000 7f91fde1ea40 -1 Falling back to public interface 2026-02-19T13:04:03.345 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:03.344+0000 7f91fde1ea40 -1 osd.1 131 log_to_monitors true 2026-02-19T13:04:03.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:04:03.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:03.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:04:03.667 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:04:03.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:03.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:04:03.829 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 135 up_thru 135 down_at 132 last_clean_interval [125,131) [v2:127.0.0.1:6810/2451325273,v1:127.0.0.1:6811/2451325273] [v2:127.0.0.1:6812/2451325273,v1:127.0.0.1:6813/2451325273] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:04:03.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:04:03.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:04:03.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:04:03.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:04:03.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:04:03.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:04:03.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:04:03.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:04:03.830 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:04:03.831 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:04:03.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:04:03.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:04:03.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:04:03.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:04:03.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:04:03.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:04:03.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:04:03.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:04:03.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:04:04.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:04:04.116 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:04:04.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:04:04.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:04.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:04:04.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=558345748484 2026-02-19T13:04:04.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 558345748484 2026-02-19T13:04:04.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-558345748484' 2026-02-19T13:04:04.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:04.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:04:04.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=579820584962 2026-02-19T13:04:04.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 579820584962 2026-02-19T13:04:04.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-558345748484 1-579820584962' 2026-02-19T13:04:04.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:04.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-558345748484 2026-02-19T13:04:04.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:04.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:04:04.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-558345748484 2026-02-19T13:04:04.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:04.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=558345748484 2026-02-19T13:04:04.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 558345748484' 2026-02-19T13:04:04.271 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 558345748484 2026-02-19T13:04:04.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:04.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 558345748482 -lt 558345748484 2026-02-19T13:04:04.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:04:05.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:04:05.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:05.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 558345748484 -lt 558345748484 2026-02-19T13:04:05.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:05.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-579820584962 2026-02-19T13:04:05.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:05.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:04:05.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-579820584962 2026-02-19T13:04:05.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:05.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=579820584962 2026-02-19T13:04:05.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 579820584962' 2026-02-19T13:04:05.593 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 579820584962 2026-02-19T13:04:05.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:04:05.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 579820584962 -lt 579820584962 2026-02-19T13:04:05.764 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:04:05.764 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:05.764 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:05.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:04:05.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:04:05.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:04:05.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:04:05.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:04:05.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:04:05.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:04:05.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:04:06.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:04:06.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:04:06.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:06.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:06.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:04:06.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:04:06.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:04:06.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1168: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/bad-val 2026-02-19T13:04:06.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:04:06.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ15 2026-02-19T13:04:06.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 15 % 2 2026-02-19T13:04:06.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=1 2026-02-19T13:04:06.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:04:06.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1172: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ15 rm-attr _ 2026-02-19T13:04:06.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:04:06.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:04:06.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:04:06.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:04:06.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ15 rm-attr _ 2026-02-19T13:04:06.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:04:06.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:04:06.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:04:06.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:04:06.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:04:06.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:04:06.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:04:06.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:04:06.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:04:06.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:04:06.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:04:06.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ15 rm-attr _ 2026-02-19T13:04:06.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:04:06.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:04:06.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:04:06.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:04:06.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:04:06.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ15 rm-attr _ 2026-02-19T13:04:06.822 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-19T13:04:06.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:04:06.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:04:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:04:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:04:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:04:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:04:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:04:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:04:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:04:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:04:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:04:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:04:06.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:04:06.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:04:06.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:04:06.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:04:06.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:04:06.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:04:06.923 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:04:06.923 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:06.923 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:06.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:06.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:06.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:04:06.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:04:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:04:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:04:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:04:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:04:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:04:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:04:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:04:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:04:06.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:04:06.925 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:04:06.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:04:06.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:04:06.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:04:06.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:04:06.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:04:06.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:04:06.942 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:06.939+0000 7f2e440d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:06.944 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:06.943+0000 7f2e440d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:06.944 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:06.943+0000 7f2e440d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:07.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:04:07.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:04:07.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:04:07.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:04:07.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:04:07.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:07.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:04:07.083 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:04:07.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:07.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:04:07.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:04:07.300 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:07.299+0000 7f2e440d1a40 -1 Falling back to public interface 2026-02-19T13:04:07.943 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:07.943+0000 7f2e440d1a40 -1 osd.1 137 log_to_monitors true 2026-02-19T13:04:08.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:04:08.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:08.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:04:08.246 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:04:08.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:08.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:04:08.404 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 141 up_thru 141 down_at 138 last_clean_interval [135,137) [v2:127.0.0.1:6810/1130350948,v1:127.0.0.1:6811/1130350948] [v2:127.0.0.1:6812/1130350948,v1:127.0.0.1:6813/1130350948] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:04:08.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:04:08.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:04:08.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:04:08.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:04:08.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:04:08.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:04:08.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:04:08.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:04:08.406 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:04:08.406 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:04:08.406 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:04:08.406 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:04:08.406 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:04:08.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:04:08.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:04:08.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:04:08.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:04:08.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:04:08.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:04:08.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:04:08.681 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:04:08.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:04:08.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:08.682 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:04:08.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=558345748485 2026-02-19T13:04:08.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 558345748485 2026-02-19T13:04:08.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-558345748485' 2026-02-19T13:04:08.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:08.754 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:04:08.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=605590388738 2026-02-19T13:04:08.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 605590388738 2026-02-19T13:04:08.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-558345748485 1-605590388738' 2026-02-19T13:04:08.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:08.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-558345748485 2026-02-19T13:04:08.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:08.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:04:08.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-558345748485 2026-02-19T13:04:08.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:08.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=558345748485 2026-02-19T13:04:08.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 558345748485' 2026-02-19T13:04:08.830 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 558345748485 2026-02-19T13:04:08.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:08.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 558345748484 -lt 558345748485 2026-02-19T13:04:08.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:04:09.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:04:09.987 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:10.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 558345748484 -lt 558345748485 2026-02-19T13:04:10.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:04:11.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:04:11.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:11.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 558345748486 -lt 558345748485 2026-02-19T13:04:11.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:11.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-605590388738 2026-02-19T13:04:11.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:11.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:04:11.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-605590388738 2026-02-19T13:04:11.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:11.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=605590388738 2026-02-19T13:04:11.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 605590388738' 2026-02-19T13:04:11.315 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 605590388738 2026-02-19T13:04:11.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:04:11.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 605590388738 -lt 605590388738 2026-02-19T13:04:11.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:04:11.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:11.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:11.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:04:11.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:04:11.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:04:11.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:04:11.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:04:11.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:04:11.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:04:11.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:04:11.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:04:11.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:04:11.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:11.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:12.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:04:12.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:04:12.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:04:12.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:04:12.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ16 2026-02-19T13:04:12.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 16 % 2 2026-02-19T13:04:12.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=0 2026-02-19T13:04:12.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:04:12.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1176: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ16 rm-attr snapset 2026-02-19T13:04:12.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:04:12.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:04:12.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:04:12.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:04:12.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ16 rm-attr snapset 2026-02-19T13:04:12.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:04:12.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:04:12.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:04:12.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:04:12.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:04:12.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:04:12.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:04:12.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:04:12.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:04:12.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:04:12.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:04:12.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ16 rm-attr snapset 2026-02-19T13:04:12.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:04:12.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:04:12.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:04:12.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:04:12.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:04:12.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ16 rm-attr snapset 2026-02-19T13:04:12.524 INFO:tasks.workunit.client.0.trial144.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-19T13:04:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:04:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:04:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:04:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:04:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:04:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:04:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:04:12.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:04:12.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:04:12.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:04:12.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:04:12.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:04:12.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:04:12.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:04:12.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:04:12.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:04:12.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:04:12.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:04:12.630 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:04:12.630 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:12.631 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:12.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:12.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:12.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:04:12.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:04:12.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:04:12.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:04:12.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:04:12.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:04:12.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:04:12.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:04:12.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:04:12.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:04:12.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:04:12.633 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:04:12.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:04:12.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:04:12.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:04:12.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:04:12.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:04:12.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:04:12.650 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:12.647+0000 7f407e4dda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:12.652 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:12.651+0000 7f407e4dda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:12.652 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:12.651+0000 7f407e4dda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:12.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:04:12.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:04:12.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:04:12.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:04:12.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:04:12.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:12.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:04:12.794 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:04:12.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:12.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:04:12.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:04:13.092 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:13.091+0000 7f407e4dda40 -1 Falling back to public interface 2026-02-19T13:04:13.700 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:13.699+0000 7f407e4dda40 -1 osd.0 142 log_to_monitors true 2026-02-19T13:04:13.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:04:13.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:13.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:04:13.954 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:04:13.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:13.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:04:14.109 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 146 up_thru 146 down_at 143 last_clean_interval [130,142) [v2:127.0.0.1:6802/4085426694,v1:127.0.0.1:6803/4085426694] [v2:127.0.0.1:6804/4085426694,v1:127.0.0.1:6805/4085426694] exists,up 0250c601-08cf-4db7-8be3-271bfebc9558 2026-02-19T13:04:14.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:04:14.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:04:14.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:04:14.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:04:14.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:04:14.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:04:14.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:04:14.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:04:14.111 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:04:14.111 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:04:14.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:04:14.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:04:14.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:04:14.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:04:14.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:04:14.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:04:14.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:04:14.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:04:14.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:04:14.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:04:14.393 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:04:14.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:04:14.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:14.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:04:14.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=627065225218 2026-02-19T13:04:14.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 627065225218 2026-02-19T13:04:14.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-627065225218' 2026-02-19T13:04:14.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:14.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:04:14.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=605590388740 2026-02-19T13:04:14.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 605590388740 2026-02-19T13:04:14.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-627065225218 1-605590388740' 2026-02-19T13:04:14.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:14.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-627065225218 2026-02-19T13:04:14.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:14.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:04:14.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-627065225218 2026-02-19T13:04:14.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:14.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=627065225218 2026-02-19T13:04:14.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 627065225218' 2026-02-19T13:04:14.546 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 627065225218 2026-02-19T13:04:14.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:14.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 627065225218 2026-02-19T13:04:14.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:04:15.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:04:15.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:15.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 627065225218 2026-02-19T13:04:15.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:04:16.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:04:16.868 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:17.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 627065225218 -lt 627065225218 2026-02-19T13:04:17.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:17.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-605590388740 2026-02-19T13:04:17.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:17.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:04:17.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-605590388740 2026-02-19T13:04:17.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:17.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=605590388740 2026-02-19T13:04:17.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 605590388740' 2026-02-19T13:04:17.030 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 605590388740 2026-02-19T13:04:17.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:04:17.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 605590388740 -lt 605590388740 2026-02-19T13:04:17.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:04:17.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:17.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:17.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:04:17.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:04:17.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:04:17.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:04:17.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:04:17.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:04:17.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:04:17.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:04:17.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:04:17.550 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:04:17.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:17.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:17.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:04:17.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:04:17.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:04:17.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1177: TEST_corrupt_scrub_replicated: echo -n bad-val 2026-02-19T13:04:17.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1178: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-19T13:04:17.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:04:17.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:04:17.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:04:17.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:04:17.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-19T13:04:17.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:04:17.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:04:17.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:04:17.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:04:17.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:04:17.744 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:04:17.744 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:04:17.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:04:17.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:04:17.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:04:18.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:04:18.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-19T13:04:18.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:04:18.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:04:18.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:04:18.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:04:18.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:04:18.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-19T13:04:18.223 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-19T13:04:18.223 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-19T13:04:18.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:04:18.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:04:18.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:04:18.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:04:18.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:04:18.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:04:18.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:04:18.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:04:18.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:04:18.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:04:18.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:04:18.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:04:18.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:04:18.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:04:18.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:04:18.308 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:04:18.308 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:04:18.308 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:04:18.308 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:04:18.309 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:18.309 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:18.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:18.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:18.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:04:18.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:04:18.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:04:18.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:04:18.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:04:18.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:04:18.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:04:18.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:04:18.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:04:18.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:04:18.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:04:18.310 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:04:18.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:04:18.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:04:18.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:04:18.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:04:18.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:04:18.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:04:18.330 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:18.327+0000 7f2e444c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:18.331 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:18.331+0000 7f2e444c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:18.332 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:18.331+0000 7f2e444c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:18.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:04:18.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:04:18.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:04:18.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:04:18.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:04:18.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:18.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:04:18.473 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:04:18.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:18.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:04:18.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:04:18.680 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:18.679+0000 7f2e444c0a40 -1 Falling back to public interface 2026-02-19T13:04:19.311 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:19.311+0000 7f2e444c0a40 -1 osd.1 147 log_to_monitors true 2026-02-19T13:04:19.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:04:19.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:19.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:04:19.632 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:04:19.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:19.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:04:19.787 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 151 up_thru 151 down_at 148 last_clean_interval [141,147) [v2:127.0.0.1:6810/984355918,v1:127.0.0.1:6811/984355918] [v2:127.0.0.1:6812/984355918,v1:127.0.0.1:6813/984355918] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:04:19.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:04:19.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:04:19.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:04:19.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:04:19.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:04:19.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:04:19.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:04:19.788 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:04:19.789 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:04:19.790 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:04:19.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:04:19.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:04:19.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:04:19.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:04:19.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:04:19.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:04:19.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:04:19.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:04:19.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:04:20.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:04:20.061 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:04:20.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:04:20.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:20.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:04:20.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=627065225220 2026-02-19T13:04:20.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 627065225220 2026-02-19T13:04:20.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-627065225220' 2026-02-19T13:04:20.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:20.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:04:20.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=648540061698 2026-02-19T13:04:20.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 648540061698 2026-02-19T13:04:20.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-627065225220 1-648540061698' 2026-02-19T13:04:20.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:20.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-627065225220 2026-02-19T13:04:20.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:20.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:04:20.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-627065225220 2026-02-19T13:04:20.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:20.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=627065225220 2026-02-19T13:04:20.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 627065225220' 2026-02-19T13:04:20.210 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 627065225220 2026-02-19T13:04:20.210 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:20.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 627065225218 -lt 627065225220 2026-02-19T13:04:20.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:04:21.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:04:21.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:21.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 627065225220 -lt 627065225220 2026-02-19T13:04:21.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:21.538 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-648540061698 2026-02-19T13:04:21.538 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:21.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:04:21.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-648540061698 2026-02-19T13:04:21.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:21.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=648540061698 2026-02-19T13:04:21.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 648540061698' 2026-02-19T13:04:21.543 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 648540061698 2026-02-19T13:04:21.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:04:21.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 648540061698 -lt 648540061698 2026-02-19T13:04:21.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:04:21.704 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:21.704 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:21.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:04:21.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:04:21.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:04:21.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:04:21.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:04:21.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:04:21.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:04:21.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:04:22.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:04:22.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:04:22.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:22.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:22.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:04:22.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:04:22.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:04:22.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:04:22.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ17 2026-02-19T13:04:22.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 17 % 2 2026-02-19T13:04:22.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=1 2026-02-19T13:04:22.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:04:22.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1183: TEST_corrupt_scrub_replicated: local payload=ROBJ17 2026-02-19T13:04:22.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: echo ROBJ17 2026-02-19T13:04:22.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1185: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:04:22.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:04:22.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:04:22.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:04:22.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:04:22.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:04:22.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:04:22.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:04:22.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:04:22.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:04:22.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:04:22.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:04:22.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:04:22.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:04:22.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:04:22.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:04:22.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:04:22.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:04:22.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:04:22.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:04:22.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:04:22.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:04:22.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:04:22.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:04:22.706 INFO:tasks.workunit.client.0.trial144.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:04:22.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:04:22.804 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:04:22.804 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:22.804 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:22.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:22.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:22.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:04:22.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:04:22.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:04:22.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:04:22.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:04:22.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:04:22.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:04:22.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:04:22.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:04:22.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:04:22.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:04:22.808 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:04:22.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:04:22.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:04:22.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:04:22.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:04:22.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:04:22.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:04:22.825 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:22.823+0000 7f8c000eaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:22.827 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:22.827+0000 7f8c000eaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:22.828 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:22.827+0000 7f8c000eaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:22.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:04:22.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:04:22.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:04:22.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:04:22.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:04:22.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:22.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:04:22.965 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:04:22.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:22.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:04:23.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:04:23.176 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:23.175+0000 7f8c000eaa40 -1 Falling back to public interface 2026-02-19T13:04:23.837 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:23.835+0000 7f8c000eaa40 -1 osd.0 152 log_to_monitors true 2026-02-19T13:04:24.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:04:24.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:24.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:04:24.127 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:04:24.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:24.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:04:24.285 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 156 up_thru 156 down_at 153 last_clean_interval [146,152) [v2:127.0.0.1:6802/3202690179,v1:127.0.0.1:6803/3202690179] [v2:127.0.0.1:6804/3202690179,v1:127.0.0.1:6805/3202690179] exists,up 0250c601-08cf-4db7-8be3-271bfebc9558 2026-02-19T13:04:24.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:04:24.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:04:24.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:04:24.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:04:24.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:04:24.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:04:24.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:04:24.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:04:24.287 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:04:24.287 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:04:24.287 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:04:24.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:04:24.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:04:24.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:04:24.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:04:24.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:04:24.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:04:24.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:04:24.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:04:24.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:04:24.557 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:04:24.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:04:24.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:24.557 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:04:24.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=670014898178 2026-02-19T13:04:24.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 670014898178 2026-02-19T13:04:24.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-670014898178' 2026-02-19T13:04:24.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:24.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:04:24.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=648540061700 2026-02-19T13:04:24.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 648540061700 2026-02-19T13:04:24.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-670014898178 1-648540061700' 2026-02-19T13:04:24.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:24.702 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-670014898178 2026-02-19T13:04:24.702 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:24.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:04:24.704 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-670014898178 2026-02-19T13:04:24.704 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:24.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=670014898178 2026-02-19T13:04:24.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 670014898178' 2026-02-19T13:04:24.706 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 670014898178 2026-02-19T13:04:24.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:24.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 670014898178 2026-02-19T13:04:24.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:04:25.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:04:25.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:26.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 670014898178 2026-02-19T13:04:26.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:04:27.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:04:27.039 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:27.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 670014898178 -lt 670014898178 2026-02-19T13:04:27.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:27.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-648540061700 2026-02-19T13:04:27.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:27.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:04:27.200 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-648540061700 2026-02-19T13:04:27.200 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:27.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=648540061700 2026-02-19T13:04:27.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 648540061700' 2026-02-19T13:04:27.202 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 648540061700 2026-02-19T13:04:27.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:04:27.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 648540061700 -lt 648540061700 2026-02-19T13:04:27.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:04:27.369 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:27.369 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:27.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:04:27.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:04:27.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:04:27.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:04:27.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:04:27.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:04:27.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:04:27.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:04:27.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:04:27.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:04:27.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:27.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:27.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:04:27.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:04:27.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:04:27.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1186: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:04:27.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:04:27.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:04:27.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:04:27.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:04:27.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:04:27.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:04:27.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:04:27.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:04:27.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:04:27.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:04:27.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:04:27.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:04:27.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:04:27.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:04:27.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:04:28.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:04:28.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:04:28.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:04:28.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:04:28.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:04:28.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:04:28.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:04:28.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:04:28.384 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-19T13:04:28.384 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:04:28.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:04:28.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:04:28.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:04:28.492 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:04:28.492 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:28.492 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:28.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:28.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:28.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:04:28.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:04:28.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:04:28.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:04:28.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:04:28.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:04:28.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:04:28.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:04:28.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:04:28.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:04:28.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:04:28.494 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:04:28.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:04:28.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:04:28.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:04:28.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:04:28.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:04:28.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:04:28.514 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:28.511+0000 7fbd8b951a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:28.515 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:28.515+0000 7fbd8b951a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:28.516 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:28.515+0000 7fbd8b951a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:28.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:04:28.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:04:28.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:04:28.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:04:28.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:04:28.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:28.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:04:28.654 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:04:28.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:28.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:04:28.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:04:28.908 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:28.907+0000 7fbd8b951a40 -1 Falling back to public interface 2026-02-19T13:04:29.519 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:29.519+0000 7fbd8b951a40 -1 osd.1 157 log_to_monitors true 2026-02-19T13:04:29.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:04:29.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:29.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:04:29.815 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:04:29.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:29.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:04:29.971 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 161 up_thru 161 down_at 158 last_clean_interval [151,157) [v2:127.0.0.1:6810/2425628961,v1:127.0.0.1:6811/2425628961] [v2:127.0.0.1:6812/2425628961,v1:127.0.0.1:6813/2425628961] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:04:29.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:04:29.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:04:29.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:04:29.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:04:29.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:04:29.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:04:29.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:04:29.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:04:29.973 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:04:29.973 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:04:29.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:04:29.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:04:29.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:04:30.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:04:30.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:04:30.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:04:30.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:04:30.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:04:30.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:04:30.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:04:30.251 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:04:30.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:04:30.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:30.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:04:30.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=670014898180 2026-02-19T13:04:30.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 670014898180 2026-02-19T13:04:30.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-670014898180' 2026-02-19T13:04:30.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:30.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:04:30.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=691489734658 2026-02-19T13:04:30.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 691489734658 2026-02-19T13:04:30.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-670014898180 1-691489734658' 2026-02-19T13:04:30.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:30.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-670014898180 2026-02-19T13:04:30.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:30.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:04:30.396 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-670014898180 2026-02-19T13:04:30.396 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:30.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=670014898180 2026-02-19T13:04:30.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 670014898180' 2026-02-19T13:04:30.398 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 670014898180 2026-02-19T13:04:30.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:30.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 670014898180 -lt 670014898180 2026-02-19T13:04:30.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:30.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-691489734658 2026-02-19T13:04:30.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:30.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:04:30.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-691489734658 2026-02-19T13:04:30.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:30.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=691489734658 2026-02-19T13:04:30.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 691489734658' 2026-02-19T13:04:30.565 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 691489734658 2026-02-19T13:04:30.566 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:04:30.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 691489734658 -lt 691489734658 2026-02-19T13:04:30.734 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:04:30.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:30.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:30.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:04:30.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:04:30.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:04:30.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:04:30.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:04:30.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:04:30.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:04:30.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:04:31.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:04:31.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:04:31.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:31.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:31.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:04:31.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:04:31.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:04:31.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:04:31.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ18 2026-02-19T13:04:31.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 18 % 2 2026-02-19T13:04:31.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=0 2026-02-19T13:04:31.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:04:31.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1191: TEST_corrupt_scrub_replicated: local payload=ROBJ18 2026-02-19T13:04:31.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1192: TEST_corrupt_scrub_replicated: echo ROBJ18 2026-02-19T13:04:31.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1193: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:04:31.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:04:31.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:04:31.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:04:31.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:04:31.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:04:31.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:04:31.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:04:31.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:04:31.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:04:31.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:04:31.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:04:31.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:04:31.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:04:31.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:04:31.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:04:31.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:04:31.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:04:31.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:04:31.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:04:31.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:04:31.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:04:31.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:04:31.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:04:31.743 INFO:tasks.workunit.client.0.trial144.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:04:31.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:04:31.856 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:04:31.856 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:31.856 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:31.856 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:31.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:31.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:04:31.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:04:31.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:04:31.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:04:31.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:04:31.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:04:31.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:04:31.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:04:31.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:04:31.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:04:31.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:04:31.858 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:04:31.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:04:31.858 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:04:31.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:04:31.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:04:31.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:04:31.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:04:31.877 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:31.875+0000 7f65211d2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:31.879 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:31.879+0000 7f65211d2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:31.880 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:31.879+0000 7f65211d2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:32.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:04:32.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:04:32.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:04:32.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:04:32.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:04:32.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:32.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:04:32.015 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:04:32.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:32.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:04:32.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:04:32.252 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:32.251+0000 7f65211d2a40 -1 Falling back to public interface 2026-02-19T13:04:32.917 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:32.915+0000 7f65211d2a40 -1 osd.0 162 log_to_monitors true 2026-02-19T13:04:33.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:04:33.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:33.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:04:33.180 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:04:33.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:33.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:04:33.335 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 166 up_thru 166 down_at 163 last_clean_interval [156,162) [v2:127.0.0.1:6802/989280484,v1:127.0.0.1:6803/989280484] [v2:127.0.0.1:6804/989280484,v1:127.0.0.1:6805/989280484] exists,up 0250c601-08cf-4db7-8be3-271bfebc9558 2026-02-19T13:04:33.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:04:33.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:04:33.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:04:33.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:04:33.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:04:33.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:04:33.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:04:33.335 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:04:33.336 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:04:33.336 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:04:33.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:04:33.337 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:04:33.337 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:04:33.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:04:33.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:04:33.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:04:33.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:04:33.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:04:33.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:04:33.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:04:33.608 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:04:33.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:04:33.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:33.608 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:04:33.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=712964571138 2026-02-19T13:04:33.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 712964571138 2026-02-19T13:04:33.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571138' 2026-02-19T13:04:33.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:33.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:04:33.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=691489734659 2026-02-19T13:04:33.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 691489734659 2026-02-19T13:04:33.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571138 1-691489734659' 2026-02-19T13:04:33.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:33.759 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-712964571138 2026-02-19T13:04:33.759 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:33.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:04:33.762 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-712964571138 2026-02-19T13:04:33.762 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:33.764 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=712964571138 2026-02-19T13:04:33.764 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 712964571138' 2026-02-19T13:04:33.764 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 712964571138 2026-02-19T13:04:33.764 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:33.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 670014898180 -lt 712964571138 2026-02-19T13:04:33.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:04:34.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:04:34.932 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:35.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571138 -lt 712964571138 2026-02-19T13:04:35.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:35.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-691489734659 2026-02-19T13:04:35.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:35.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:04:35.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-691489734659 2026-02-19T13:04:35.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:35.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=691489734659 2026-02-19T13:04:35.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 691489734659' 2026-02-19T13:04:35.094 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 691489734659 2026-02-19T13:04:35.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:04:35.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 691489734659 -lt 691489734659 2026-02-19T13:04:35.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:04:35.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:35.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:35.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:04:35.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:04:35.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:04:35.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:04:35.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:04:35.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:04:35.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:04:35.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:04:35.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:04:35.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:04:35.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:35.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:35.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:04:35.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:04:35.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:04:35.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1194: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:04:35.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:04:35.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:04:35.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:04:35.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:04:35.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:04:35.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:04:35.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:04:35.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:04:35.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:04:35.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:04:35.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:04:35.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:04:35.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:04:35.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:04:35.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:04:36.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:04:36.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:04:36.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:04:36.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:04:36.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:04:36.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:04:36.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:04:36.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:04:36.335 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-19T13:04:36.335 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-19T13:04:36.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:04:36.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:04:36.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:04:36.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:04:36.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:04:36.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:04:36.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:04:36.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:04:36.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:04:36.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:04:36.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:04:36.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:04:36.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:04:36.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:04:36.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:04:36.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:04:36.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:04:36.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:04:36.445 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:04:36.445 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:36.445 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:36.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:36.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:36.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:04:36.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:04:36.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:04:36.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:04:36.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:04:36.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:04:36.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:04:36.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:04:36.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:04:36.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:04:36.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:04:36.446 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:04:36.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:04:36.446 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:04:36.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:04:36.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:04:36.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:04:36.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:04:36.464 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:36.463+0000 7f3c22d77a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:36.465 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:36.463+0000 7f3c22d77a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:36.466 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:36.467+0000 7f3c22d77a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:36.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:04:36.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:04:36.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:04:36.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:04:36.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:04:36.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:36.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:04:36.606 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:04:36.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:36.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:04:36.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:04:36.812 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:36.811+0000 7f3c22d77a40 -1 Falling back to public interface 2026-02-19T13:04:37.456 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:37.455+0000 7f3c22d77a40 -1 osd.1 168 log_to_monitors true 2026-02-19T13:04:37.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:04:37.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:37.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:04:37.770 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:04:37.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:37.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:04:37.927 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 172 up_thru 172 down_at 169 last_clean_interval [161,168) [v2:127.0.0.1:6810/686351625,v1:127.0.0.1:6811/686351625] [v2:127.0.0.1:6812/686351625,v1:127.0.0.1:6813/686351625] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:04:37.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:04:37.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:04:37.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:04:37.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:04:37.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:04:37.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:04:37.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:04:37.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:04:37.929 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:04:37.929 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:04:37.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:04:37.930 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:04:37.930 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:04:38.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:04:38.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:04:38.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:04:38.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:04:38.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:04:38.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:04:38.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:04:38.210 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:04:38.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:04:38.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:38.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:04:38.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=712964571140 2026-02-19T13:04:38.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 712964571140 2026-02-19T13:04:38.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571140' 2026-02-19T13:04:38.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:38.284 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:04:38.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=738734374914 2026-02-19T13:04:38.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 738734374914 2026-02-19T13:04:38.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571140 1-738734374914' 2026-02-19T13:04:38.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:38.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-712964571140 2026-02-19T13:04:38.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:38.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:04:38.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-712964571140 2026-02-19T13:04:38.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:38.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=712964571140 2026-02-19T13:04:38.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 712964571140' 2026-02-19T13:04:38.362 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 712964571140 2026-02-19T13:04:38.363 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:38.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571140 -lt 712964571140 2026-02-19T13:04:38.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:38.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-738734374914 2026-02-19T13:04:38.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:38.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:04:38.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-738734374914 2026-02-19T13:04:38.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:38.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=738734374914 2026-02-19T13:04:38.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 738734374914' 2026-02-19T13:04:38.526 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 738734374914 2026-02-19T13:04:38.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:04:38.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 738734374914 -lt 738734374914 2026-02-19T13:04:38.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:04:38.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:38.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:38.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:04:38.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:04:38.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:04:38.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:04:38.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:04:38.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:04:38.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:04:38.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:04:39.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:04:39.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:04:39.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:39.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:39.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:04:39.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:04:39.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:04:39.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1196: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ18 corrupt-info 2026-02-19T13:04:39.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:04:39.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:04:39.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:04:39.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:04:39.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ18 corrupt-info 2026-02-19T13:04:39.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:04:39.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:04:39.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:04:39.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:04:39.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:04:39.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:04:39.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:04:39.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:04:39.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:04:39.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:04:39.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:04:39.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ18 corrupt-info 2026-02-19T13:04:39.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:04:39.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:04:39.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:04:39.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:04:39.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:04:39.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ18 corrupt-info 2026-02-19T13:04:39.714 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-19T13:04:39.714 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-19T13:04:39.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:04:39.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:04:39.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:04:39.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:04:39.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:04:39.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:04:39.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:04:39.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:04:39.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:04:39.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:04:39.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:04:39.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:04:39.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:04:39.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:04:39.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:04:39.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:04:39.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:04:39.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:04:39.809 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:04:39.809 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:39.809 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:39.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:39.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:39.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:04:39.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:04:39.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:04:39.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:04:39.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:04:39.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:04:39.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:04:39.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:04:39.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:04:39.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:04:39.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:04:39.810 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:04:39.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:04:39.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:04:39.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:04:39.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:04:39.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:04:39.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:04:39.827 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:39.827+0000 7f7cd7a45a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:39.829 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:39.827+0000 7f7cd7a45a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:39.830 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:39.827+0000 7f7cd7a45a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:39.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:04:39.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:04:39.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:04:39.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:04:39.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:04:39.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:39.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:04:39.967 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:04:39.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:39.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:04:40.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:04:40.248 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:40.247+0000 7f7cd7a45a40 -1 Falling back to public interface 2026-02-19T13:04:40.801 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:40.799+0000 7f7cd7a45a40 -1 osd.1 173 log_to_monitors true 2026-02-19T13:04:41.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:04:41.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:41.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:04:41.130 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:04:41.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:41.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:04:41.286 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 177 up_thru 177 down_at 174 last_clean_interval [172,173) [v2:127.0.0.1:6810/3473231059,v1:127.0.0.1:6811/3473231059] [v2:127.0.0.1:6812/3473231059,v1:127.0.0.1:6813/3473231059] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:04:41.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:04:41.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:04:41.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:04:41.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:04:41.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:04:41.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:04:41.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:04:41.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:04:41.288 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:04:41.288 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:04:41.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:04:41.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:04:41.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:04:41.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:04:41.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:04:41.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:04:41.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:04:41.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:04:41.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:04:41.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:04:41.563 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:04:41.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:04:41.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:41.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:04:41.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=712964571141 2026-02-19T13:04:41.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 712964571141 2026-02-19T13:04:41.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571141' 2026-02-19T13:04:41.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:41.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:04:41.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=760209211394 2026-02-19T13:04:41.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 760209211394 2026-02-19T13:04:41.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571141 1-760209211394' 2026-02-19T13:04:41.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:41.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-712964571141 2026-02-19T13:04:41.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:41.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:04:41.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-712964571141 2026-02-19T13:04:41.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:41.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=712964571141 2026-02-19T13:04:41.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 712964571141' 2026-02-19T13:04:41.715 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 712964571141 2026-02-19T13:04:41.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:41.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571140 -lt 712964571141 2026-02-19T13:04:41.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:04:42.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:04:42.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:43.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571141 -lt 712964571141 2026-02-19T13:04:43.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:43.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-760209211394 2026-02-19T13:04:43.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:43.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:04:43.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-760209211394 2026-02-19T13:04:43.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:43.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=760209211394 2026-02-19T13:04:43.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 760209211394' 2026-02-19T13:04:43.042 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 760209211394 2026-02-19T13:04:43.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:04:43.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 760209211394 -lt 760209211394 2026-02-19T13:04:43.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:04:43.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:43.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:43.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:04:43.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:04:43.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:04:43.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:04:43.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:04:43.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:04:43.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:04:43.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:04:43.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:04:43.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:04:43.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:43.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:43.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:04:43.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:04:43.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:04:43.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-19T13:04:43.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: objname=ROBJ19 2026-02-19T13:04:43.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: expr 19 % 2 2026-02-19T13:04:43.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1092: TEST_corrupt_scrub_replicated: local osd=1 2026-02-19T13:04:43.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: case $i in 2026-02-19T13:04:43.748 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1205: TEST_corrupt_scrub_replicated: get_pg csr_pool ROBJ0 2026-02-19T13:04:43.748 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=csr_pool 2026-02-19T13:04:43.748 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=ROBJ0 2026-02-19T13:04:43.748 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map csr_pool ROBJ0 2026-02-19T13:04:43.748 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:04:43.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1205: TEST_corrupt_scrub_replicated: local pg=3.0 2026-02-19T13:04:43.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1207: TEST_corrupt_scrub_replicated: ceph tell 'osd.*' injectargs -- --osd-max-object-size=1048576 2026-02-19T13:04:43.979 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: {} 2026-02-19T13:04:43.979 INFO:tasks.workunit.client.0.trial144.stderr:osd.0: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_max_object_size = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2026-02-19T13:04:43.988 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: {} 2026-02-19T13:04:43.988 INFO:tasks.workunit.client.0.trial144.stderr:osd.1: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_max_object_size = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2026-02-19T13:04:43.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1209: TEST_corrupt_scrub_replicated: inject_eio rep data csr_pool ROBJ11 td/osd-scrub-repair 0 2026-02-19T13:04:43.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=rep 2026-02-19T13:04:43.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:04:43.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:04:43.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:04:43.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=csr_pool 2026-02-19T13:04:43.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:04:43.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=ROBJ11 2026-02-19T13:04:43.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:04:43.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/osd-scrub-repair 2026-02-19T13:04:43.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:04:43.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-19T13:04:43.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:04:43.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds csr_pool ROBJ11 2026-02-19T13:04:43.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=csr_pool 2026-02-19T13:04:43.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=ROBJ11 2026-02-19T13:04:43.998 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map csr_pool ROBJ11 2026-02-19T13:04:43.998 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:04:44.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-19T13:04:44.154 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:04:44.155 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-19T13:04:44.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('1' '0') 2026-02-19T13:04:44.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:04:44.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-19T13:04:44.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' rep '!=' ec ']' 2026-02-19T13:04:44.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2497: inject_eio: shard_id= 2026-02-19T13:04:44.155 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/osd-scrub-repair/1/type 2026-02-19T13:04:44.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:04:44.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T13:04:44.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:04:44.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T13:04:44.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:04:44.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:04:44.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:04:44.157 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T13:04:44.158 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:04:44.158 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:04:44.158 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:04:44.158 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:44.158 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:44.158 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T13:04:44.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:04:44.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:04:44.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:04:44.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:04:44.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-19T13:04:44.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:04:44.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:04:44.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:04:44.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:44.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:44.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T13:04:44.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:04:44.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.35818/ceph-osd.1.asok injectdataerr csr_pool ROBJ11 2026-02-19T13:04:44.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1210: TEST_corrupt_scrub_replicated: inject_eio rep mdata csr_pool ROBJ12 td/osd-scrub-repair 1 2026-02-19T13:04:44.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=rep 2026-02-19T13:04:44.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:04:44.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=mdata 2026-02-19T13:04:44.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:04:44.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=csr_pool 2026-02-19T13:04:44.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:04:44.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=ROBJ12 2026-02-19T13:04:44.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:04:44.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/osd-scrub-repair 2026-02-19T13:04:44.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:04:44.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-19T13:04:44.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:04:44.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds csr_pool ROBJ12 2026-02-19T13:04:44.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=csr_pool 2026-02-19T13:04:44.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=ROBJ12 2026-02-19T13:04:44.282 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map csr_pool ROBJ12 2026-02-19T13:04:44.283 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:04:44.444 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-19T13:04:44.444 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:04:44.444 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-19T13:04:44.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('1' '0') 2026-02-19T13:04:44.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:04:44.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=0 2026-02-19T13:04:44.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' rep '!=' ec ']' 2026-02-19T13:04:44.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2497: inject_eio: shard_id= 2026-02-19T13:04:44.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/osd-scrub-repair/0/type 2026-02-19T13:04:44.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:04:44.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-19T13:04:44.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:04:44.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=0 2026-02-19T13:04:44.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:04:44.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:04:44.447 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:04:44.447 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.0 2026-02-19T13:04:44.447 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:04:44.447 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:04:44.448 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:04:44.448 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:44.448 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:44.448 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.0.asok 2026-02-19T13:04:44.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:04:44.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:04:44.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:04:44.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:04:44.511 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.0 2026-02-19T13:04:44.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:04:44.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:04:44.512 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:04:44.512 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:44.512 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:44.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.0.asok 2026-02-19T13:04:44.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:04:44.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.35818/ceph-osd.0.asok injectmdataerr csr_pool ROBJ12 2026-02-19T13:04:44.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1211: TEST_corrupt_scrub_replicated: inject_eio rep mdata csr_pool ROBJ13 td/osd-scrub-repair 1 2026-02-19T13:04:44.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=rep 2026-02-19T13:04:44.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:04:44.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=mdata 2026-02-19T13:04:44.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:04:44.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=csr_pool 2026-02-19T13:04:44.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:04:44.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=ROBJ13 2026-02-19T13:04:44.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:04:44.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/osd-scrub-repair 2026-02-19T13:04:44.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:04:44.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-19T13:04:44.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:04:44.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds csr_pool ROBJ13 2026-02-19T13:04:44.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=csr_pool 2026-02-19T13:04:44.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=ROBJ13 2026-02-19T13:04:44.573 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map csr_pool ROBJ13 2026-02-19T13:04:44.573 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:04:44.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-19T13:04:44.736 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:04:44.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-19T13:04:44.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('1' '0') 2026-02-19T13:04:44.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:04:44.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=0 2026-02-19T13:04:44.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' rep '!=' ec ']' 2026-02-19T13:04:44.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2497: inject_eio: shard_id= 2026-02-19T13:04:44.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/osd-scrub-repair/0/type 2026-02-19T13:04:44.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:04:44.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-19T13:04:44.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:04:44.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=0 2026-02-19T13:04:44.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:04:44.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:04:44.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:04:44.739 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.0 2026-02-19T13:04:44.739 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:04:44.739 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:04:44.740 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:04:44.740 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:44.740 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:44.740 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.0.asok 2026-02-19T13:04:44.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:04:44.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:04:44.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:04:44.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:04:44.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.0 2026-02-19T13:04:44.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:04:44.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:04:44.804 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:04:44.804 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:44.804 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:44.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.0.asok 2026-02-19T13:04:44.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:04:44.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.35818/ceph-osd.0.asok injectmdataerr csr_pool ROBJ13 2026-02-19T13:04:44.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1212: TEST_corrupt_scrub_replicated: inject_eio rep data csr_pool ROBJ13 td/osd-scrub-repair 0 2026-02-19T13:04:44.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=rep 2026-02-19T13:04:44.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:04:44.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:04:44.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:04:44.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=csr_pool 2026-02-19T13:04:44.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:04:44.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=ROBJ13 2026-02-19T13:04:44.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:04:44.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/osd-scrub-repair 2026-02-19T13:04:44.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:04:44.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-19T13:04:44.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:04:44.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds csr_pool ROBJ13 2026-02-19T13:04:44.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=csr_pool 2026-02-19T13:04:44.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=ROBJ13 2026-02-19T13:04:44.865 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map csr_pool ROBJ13 2026-02-19T13:04:44.866 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:04:45.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-19T13:04:45.026 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:04:45.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-19T13:04:45.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('1' '0') 2026-02-19T13:04:45.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:04:45.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-19T13:04:45.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' rep '!=' ec ']' 2026-02-19T13:04:45.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2497: inject_eio: shard_id= 2026-02-19T13:04:45.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/osd-scrub-repair/1/type 2026-02-19T13:04:45.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:04:45.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T13:04:45.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:04:45.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T13:04:45.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:04:45.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:04:45.029 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:04:45.029 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T13:04:45.029 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:04:45.029 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:04:45.030 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:04:45.030 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:45.030 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:45.030 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T13:04:45.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:04:45.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:04:45.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:04:45.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:04:45.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-19T13:04:45.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:04:45.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:04:45.092 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:04:45.092 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:45.092 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:45.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T13:04:45.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:04:45.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.35818/ceph-osd.1.asok injectdataerr csr_pool ROBJ13 2026-02-19T13:04:45.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1214: TEST_corrupt_scrub_replicated: pg_scrub 3.0 2026-02-19T13:04:45.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=3.0 2026-02-19T13:04:45.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 3.0 2026-02-19T13:04:45.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=3.0 2026-02-19T13:04:45.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:04:45.152 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:04:45.152 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:04:45.153 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:04:45.153 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:04:45.153 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:04:45.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:04:45.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:04:45.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:04:45.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:04:45.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:04:45.466 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:04:45.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:04:45.628 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:04:45.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:04:45.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:45.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:04:45.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=712964571143 2026-02-19T13:04:45.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 712964571143 2026-02-19T13:04:45.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571143' 2026-02-19T13:04:45.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:45.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:04:45.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=760209211395 2026-02-19T13:04:45.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 760209211395 2026-02-19T13:04:45.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571143 1-760209211395' 2026-02-19T13:04:45.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:45.774 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-712964571143 2026-02-19T13:04:45.774 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:45.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:04:45.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-712964571143 2026-02-19T13:04:45.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:45.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=712964571143 2026-02-19T13:04:45.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 712964571143' 2026-02-19T13:04:45.778 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 712964571143 2026-02-19T13:04:45.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:45.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571142 -lt 712964571143 2026-02-19T13:04:45.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:04:46.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:04:46.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:47.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571143 -lt 712964571143 2026-02-19T13:04:47.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:47.098 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-760209211395 2026-02-19T13:04:47.098 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:47.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:04:47.100 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-760209211395 2026-02-19T13:04:47.100 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:47.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=760209211395 2026-02-19T13:04:47.102 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 760209211395 2026-02-19T13:04:47.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 760209211395' 2026-02-19T13:04:47.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:04:47.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 760209211396 -lt 760209211395 2026-02-19T13:04:47.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:04:47.259 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 3.0 loop 0 2026-02-19T13:04:47.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-19T13:04:47.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 3.0 2026-02-19T13:04:47.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=3.0 2026-02-19T13:04:47.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:04:47.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 3.0 query 2026-02-19T13:04:47.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:04:47.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-19T13:04:47.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:04:47.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:04:47.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:04:47.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 3.0 2026-02-19T13:04:47.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:04:47.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:04:47.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:04:47.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:04:47.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-19T13:02:55.699858+0000 2026-02-19T13:04:47.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 3.0 2026-02-19T13:04:47.637 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0 on osd.1 to scrub 2026-02-19T13:04:47.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 3.0 2026-02-19T13:02:55.699858+0000 2026-02-19T13:04:47.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:04:47.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:02:55.699858+0000 2026-02-19T13:04:47.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:04:47.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:04:47.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:04:47.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:04:47.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:04:47.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:04:47.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:04:47.651 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:04:47.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:02:55.699858+0000 '>' 2026-02-19T13:02:55.699858+0000 2026-02-19T13:04:47.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:04:48.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:04:48.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:04:48.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:04:48.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:04:48.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:04:48.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:04:48.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:04:48.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:02:55.699858+0000 '>' 2026-02-19T13:02:55.699858+0000 2026-02-19T13:04:48.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:04:49.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:04:49.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:04:49.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:04:49.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:04:49.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:04:49.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:04:49.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:04:50.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:02:55.699858+0000 '>' 2026-02-19T13:02:55.699858+0000 2026-02-19T13:04:50.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:04:51.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:04:51.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:04:51.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:04:51.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:04:51.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:04:51.133 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:04:51.133 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:04:51.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:02:55.699858+0000 '>' 2026-02-19T13:02:55.699858+0000 2026-02-19T13:04:51.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:04:52.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:04:52.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:04:52.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:04:52.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:04:52.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:04:52.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:04:52.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:04:52.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:04:47.795433+0000 '>' 2026-02-19T13:02:55.699858+0000 2026-02-19T13:04:52.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:04:52.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1216: TEST_corrupt_scrub_replicated: ERRORS=0 2026-02-19T13:04:52.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1217: TEST_corrupt_scrub_replicated: declare -a s_err_strings 2026-02-19T13:04:52.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1218: TEST_corrupt_scrub_replicated: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:30259878:::ROBJ15:head : candidate had a missing info key' 2026-02-19T13:04:52.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1219: TEST_corrupt_scrub_replicated: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:33aca486:::ROBJ18:head : object info inconsistent ' 2026-02-19T13:04:52.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1220: TEST_corrupt_scrub_replicated: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:5c7b2c47:::ROBJ16:head : candidate had a corrupt snapset' 2026-02-19T13:04:52.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1221: TEST_corrupt_scrub_replicated: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:5c7b2c47:::ROBJ16:head : candidate had a missing snapset key' 2026-02-19T13:04:52.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1222: TEST_corrupt_scrub_replicated: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:5c7b2c47:::ROBJ16:head : failed to pick suitable object info' 2026-02-19T13:04:52.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1223: TEST_corrupt_scrub_replicated: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:86586531:::ROBJ8:head : attr value mismatch '\''_key1-ROBJ8'\'', attr name mismatch '\''_key3-ROBJ8'\'', attr name mismatch '\''_key2-ROBJ8'\''' 2026-02-19T13:04:52.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1224: TEST_corrupt_scrub_replicated: err_strings[6]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:bc819597:::ROBJ12:head : candidate had a stat error' 2026-02-19T13:04:52.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1225: TEST_corrupt_scrub_replicated: err_strings[7]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:c0c86b1d:::ROBJ14:head : candidate had a missing info key' 2026-02-19T13:04:52.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1226: TEST_corrupt_scrub_replicated: err_strings[8]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:c0c86b1d:::ROBJ14:head : candidate had a corrupt info' 2026-02-19T13:04:52.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1227: TEST_corrupt_scrub_replicated: err_strings[9]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:c0c86b1d:::ROBJ14:head : failed to pick suitable object info' 2026-02-19T13:04:52.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1228: TEST_corrupt_scrub_replicated: err_strings[10]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : candidate size 9 info size 7 mismatch' 2026-02-19T13:04:52.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1229: TEST_corrupt_scrub_replicated: err_strings[11]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : size 9 != size 7 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from shard 0' 2026-02-19T13:04:52.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1230: TEST_corrupt_scrub_replicated: err_strings[12]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:d60617f9:::ROBJ13:head : candidate had a stat error' 2026-02-19T13:04:52.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1231: TEST_corrupt_scrub_replicated: err_strings[13]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 3:f2a5b2a4:::ROBJ3:head : missing' 2026-02-19T13:04:52.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1232: TEST_corrupt_scrub_replicated: err_strings[14]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ffdb2004:::ROBJ9:head : candidate size 1 info size 7 mismatch' 2026-02-19T13:04:52.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1233: TEST_corrupt_scrub_replicated: err_strings[15]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ffdb2004:::ROBJ9:head : object info inconsistent ' 2026-02-19T13:04:52.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1234: TEST_corrupt_scrub_replicated: err_strings[16]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 3:c0c86b1d:::ROBJ14:head : no '\''_'\'' attr' 2026-02-19T13:04:52.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1235: TEST_corrupt_scrub_replicated: err_strings[17]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 3:5c7b2c47:::ROBJ16:head : can'\''t decode '\''snapset'\'' attr .* v=3 cannot decode .* Malformed input' 2026-02-19T13:04:52.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1236: TEST_corrupt_scrub_replicated: err_strings[18]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub : stat mismatch, got 19/19 objects, 0/0 clones, 18/19 dirty, 18/19 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 1049713/1049720 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' 2026-02-19T13:04:52.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1237: TEST_corrupt_scrub_replicated: err_strings[19]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 1 missing, 8 inconsistent objects' 2026-02-19T13:04:52.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1238: TEST_corrupt_scrub_replicated: err_strings[20]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 18 errors' 2026-02-19T13:04:52.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1239: TEST_corrupt_scrub_replicated: err_strings[21]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:123a5f55:::ROBJ19:head : size 1049600 > 1048576 is too large' 2026-02-19T13:04:52.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:30259878:::ROBJ15:head : candidate had a missing info key' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:33aca486:::ROBJ18:head : object info inconsistent ' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:5c7b2c47:::ROBJ16:head : candidate had a corrupt snapset' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:5c7b2c47:::ROBJ16:head : candidate had a missing snapset key' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:5c7b2c47:::ROBJ16:head : failed to pick suitable object info' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:86586531:::ROBJ8:head : attr value mismatch '\''_key1-ROBJ8'\'', attr name mismatch '\''_key3-ROBJ8'\'', attr name mismatch '\''_key2-ROBJ8'\''' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:bc819597:::ROBJ12:head : candidate had a stat error' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:c0c86b1d:::ROBJ14:head : candidate had a missing info key' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:c0c86b1d:::ROBJ14:head : candidate had a corrupt info' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:c0c86b1d:::ROBJ14:head : failed to pick suitable object info' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : candidate size 9 info size 7 mismatch' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : size 9 != size 7 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from shard 0' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:d60617f9:::ROBJ13:head : candidate had a stat error' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 3:f2a5b2a4:::ROBJ3:head : missing' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ffdb2004:::ROBJ9:head : candidate size 1 info size 7 mismatch' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ffdb2004:::ROBJ9:head : object info inconsistent ' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 3:c0c86b1d:::ROBJ14:head : no '\''_'\'' attr' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 3:5c7b2c47:::ROBJ16:head : can'\''t decode '\''snapset'\'' attr .* v=3 cannot decode .* Malformed input' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub : stat mismatch, got 19/19 objects, 0/0 clones, 18/19 dirty, 18/19 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 1049713/1049720 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 1 missing, 8 inconsistent objects' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 18 errors' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1241: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:04:52.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1243: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:123a5f55:::ROBJ19:head : size 1049600 > 1048576 is too large' td/osd-scrub-repair/osd.1.log 2026-02-19T13:04:52.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1250: TEST_corrupt_scrub_replicated: rados list-inconsistent-pg csr_pool 2026-02-19T13:04:52.649 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1252: TEST_corrupt_scrub_replicated: jq '. | length' td/osd-scrub-repair/json 2026-02-19T13:04:52.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1252: TEST_corrupt_scrub_replicated: test 1 = 1 2026-02-19T13:04:52.664 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1254: TEST_corrupt_scrub_replicated: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-19T13:04:52.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1254: TEST_corrupt_scrub_replicated: test 3.0 = 3.0 2026-02-19T13:04:52.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1256: TEST_corrupt_scrub_replicated: rados list-inconsistent-obj 3.0 2026-02-19T13:04:52.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1258: TEST_corrupt_scrub_replicated: jq .epoch td/osd-scrub-repair/json 2026-02-19T13:04:52.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1258: TEST_corrupt_scrub_replicated: epoch=177 2026-02-19T13:04:52.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1260: TEST_corrupt_scrub_replicated: jq 'def walk(f): 2026-02-19T13:04:52.706 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:04:52.706 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:04:52.706 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:04:52.706 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:04:52.706 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:04:52.706 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:04:52.707 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:04:52.707 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:04:52.707 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:04:52.707 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:04:52.707 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:04:52.707 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-19T13:04:52.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1260: TEST_corrupt_scrub_replicated: jq .inconsistents 2026-02-19T13:04:52.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1260: TEST_corrupt_scrub_replicated: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:04:52.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2173: TEST_corrupt_scrub_replicated: jq 'def walk(f): 2026-02-19T13:04:52.725 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:04:52.725 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:04:52.725 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:04:52.725 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:04:52.725 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:04:52.725 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:04:52.725 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:04:52.726 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:04:52.726 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:04:52.726 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:04:52.726 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:04:52.726 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-19T13:04:52.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2173: TEST_corrupt_scrub_replicated: jq .inconsistents 2026-02-19T13:04:52.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2173: TEST_corrupt_scrub_replicated: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:04:52.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2174: TEST_corrupt_scrub_replicated: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-19T13:04:52.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-19T13:04:52.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2175: TEST_corrupt_scrub_replicated: test no = yes 2026-02-19T13:04:52.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2180: TEST_corrupt_scrub_replicated: test '' = yes 2026-02-19T13:04:52.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2185: TEST_corrupt_scrub_replicated: objname=ROBJ9 2026-02-19T13:04:52.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2187: TEST_corrupt_scrub_replicated: echo -n ZZZ 2026-02-19T13:04:52.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2188: TEST_corrupt_scrub_replicated: rados --pool csr_pool put ROBJ9 td/osd-scrub-repair/change 2026-02-19T13:04:52.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2190: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-19T13:04:52.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:04:52.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:04:52.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:04:52.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:04:52.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-19T13:04:52.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:04:52.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:04:52.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:04:52.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:04:52.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:04:52.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:04:52.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:04:52.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:04:52.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:04:52.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:04:53.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:04:53.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-19T13:04:53.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:04:53.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:04:53.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:04:53.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:04:53.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:04:53.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-19T13:04:53.238 INFO:tasks.workunit.client.0.trial144.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-19T13:04:53.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:04:53.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:04:53.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:04:53.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:04:53.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:04:53.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:04:53.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:04:53.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:04:53.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:04:53.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:04:53.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:04:53.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:04:53.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:04:53.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:04:53.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:04:53.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:04:53.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:04:53.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:04:53.326 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:04:53.326 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:53.326 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:53.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:53.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:53.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:04:53.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:04:53.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:04:53.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:04:53.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:04:53.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:04:53.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:04:53.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:04:53.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:04:53.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:04:53.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:04:53.329 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:04:53.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:04:53.329 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:04:53.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:04:53.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:04:53.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:04:53.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:04:53.346 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:53.343+0000 7f4c182c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:53.347 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:53.347+0000 7f4c182c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:53.349 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:53.347+0000 7f4c182c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:53.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:04:53.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:04:53.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:04:53.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:04:53.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:04:53.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:53.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:04:53.490 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:04:53.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:53.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:04:53.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:04:53.748 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:53.747+0000 7f4c182c7a40 -1 Falling back to public interface 2026-02-19T13:04:54.435 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:54.435+0000 7f4c182c7a40 -1 osd.0 180 log_to_monitors true 2026-02-19T13:04:54.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:04:54.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:54.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:04:54.648 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:04:54.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:54.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:04:54.808 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 183 up_thru 183 down_at 180 last_clean_interval [166,180) [v2:127.0.0.1:6802/2589676876,v1:127.0.0.1:6803/2589676876] [v2:127.0.0.1:6804/2589676876,v1:127.0.0.1:6805/2589676876] exists,up 0250c601-08cf-4db7-8be3-271bfebc9558 2026-02-19T13:04:54.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:04:54.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:04:54.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:04:54.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:04:54.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:04:54.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:04:54.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:04:54.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:04:54.810 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:04:54.810 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:04:54.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:04:54.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:04:54.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:04:54.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:04:54.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:04:54.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:04:54.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:04:54.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:04:54.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:04:55.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:04:55.074 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:04:55.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:04:55.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:55.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:04:55.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=785979015170 2026-02-19T13:04:55.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 785979015170 2026-02-19T13:04:55.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-785979015170' 2026-02-19T13:04:55.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:55.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:04:55.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=760209211398 2026-02-19T13:04:55.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 760209211398 2026-02-19T13:04:55.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-785979015170 1-760209211398' 2026-02-19T13:04:55.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:55.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-785979015170 2026-02-19T13:04:55.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:55.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:04:55.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-785979015170 2026-02-19T13:04:55.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:55.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=785979015170 2026-02-19T13:04:55.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 785979015170' 2026-02-19T13:04:55.223 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 785979015170 2026-02-19T13:04:55.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:55.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571144 -lt 785979015170 2026-02-19T13:04:55.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:04:56.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:04:56.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:56.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 785979015170 -lt 785979015170 2026-02-19T13:04:56.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:56.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-760209211398 2026-02-19T13:04:56.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:56.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:04:56.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-760209211398 2026-02-19T13:04:56.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:56.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=760209211398 2026-02-19T13:04:56.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 760209211398' 2026-02-19T13:04:56.551 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 760209211398 2026-02-19T13:04:56.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:04:56.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 760209211399 -lt 760209211398 2026-02-19T13:04:56.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:04:56.716 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:56.716 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:56.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:04:56.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:04:56.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:04:56.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:04:56.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:04:56.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:04:56.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:04:56.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:04:57.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:04:57.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:04:57.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:04:57.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:04:57.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:04:57.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:04:57.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:04:57.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2191: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/oi td/osd-scrub-repair/change 2026-02-19T13:04:57.249 INFO:tasks.workunit.client.0.trial144.stderr:rm: cannot remove 'td/osd-scrub-repair/oi': No such file or directory 2026-02-19T13:04:57.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2193: TEST_corrupt_scrub_replicated: objname=ROBJ10 2026-02-19T13:04:57.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2194: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ10 get-attr _ 2026-02-19T13:04:57.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:04:57.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:04:57.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:04:57.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:04:57.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ10 get-attr _ 2026-02-19T13:04:57.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:04:57.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:04:57.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:04:57.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:04:57.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:04:57.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:04:57.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:04:57.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:04:57.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:04:57.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:04:57.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:04:57.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ10 get-attr _ 2026-02-19T13:04:57.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:04:57.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:04:57.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:04:57.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:04:57.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:04:57.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ10 get-attr _ 2026-02-19T13:04:57.631 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-19T13:04:57.632 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-19T13:04:57.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:04:57.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:04:57.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:04:57.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:04:57.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:04:57.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:04:57.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:04:57.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:04:57.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:04:57.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:04:57.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:04:57.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:04:57.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:04:57.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:04:57.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:04:57.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:04:57.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:04:57.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:04:57.746 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:04:57.746 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:04:57.746 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:04:57.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:57.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:04:57.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:04:57.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:04:57.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:04:57.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:04:57.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:04:57.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:04:57.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:04:57.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:04:57.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:04:57.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:04:57.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:04:57.749 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:04:57.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:04:57.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:04:57.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:04:57.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:04:57.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:04:57.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:04:57.763 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:57.762+0000 7fbcccc7aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:57.765 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:57.762+0000 7fbcccc7aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:57.766 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:57.762+0000 7fbcccc7aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:04:57.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:04:57.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:04:57.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:04:57.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:04:57.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:04:57.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:57.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:04:57.905 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:04:57.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:57.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:04:58.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:04:58.088 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:58.086+0000 7fbcccc7aa40 -1 Falling back to public interface 2026-02-19T13:04:58.660 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:04:58.658+0000 7fbcccc7aa40 -1 osd.1 185 log_to_monitors true 2026-02-19T13:04:59.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:04:59.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:04:59.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:04:59.067 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:04:59.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:04:59.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:04:59.228 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 188 up_thru 188 down_at 185 last_clean_interval [177,185) [v2:127.0.0.1:6810/3157368981,v1:127.0.0.1:6811/3157368981] [v2:127.0.0.1:6812/3157368981,v1:127.0.0.1:6813/3157368981] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:04:59.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:04:59.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:04:59.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:04:59.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:04:59.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:04:59.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:04:59.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:04:59.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:04:59.229 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:04:59.230 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:04:59.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:04:59.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:04:59.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:04:59.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:04:59.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:04:59.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:04:59.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:04:59.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:04:59.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:04:59.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:04:59.501 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:04:59.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:04:59.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:59.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:04:59.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=785979015172 2026-02-19T13:04:59.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 785979015172 2026-02-19T13:04:59.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-785979015172' 2026-02-19T13:04:59.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:04:59.573 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:04:59.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=807453851650 2026-02-19T13:04:59.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 807453851650 2026-02-19T13:04:59.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-785979015172 1-807453851650' 2026-02-19T13:04:59.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:04:59.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-785979015172 2026-02-19T13:04:59.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:04:59.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:04:59.649 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-785979015172 2026-02-19T13:04:59.649 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:04:59.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=785979015172 2026-02-19T13:04:59.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 785979015172' 2026-02-19T13:04:59.651 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 785979015172 2026-02-19T13:04:59.651 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:04:59.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 785979015170 -lt 785979015172 2026-02-19T13:04:59.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:05:00.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:05:00.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:00.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 785979015172 -lt 785979015172 2026-02-19T13:05:00.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:00.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-807453851650 2026-02-19T13:05:00.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:00.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:05:00.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-807453851650 2026-02-19T13:05:00.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:00.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=807453851650 2026-02-19T13:05:00.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 807453851650' 2026-02-19T13:05:00.976 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 807453851650 2026-02-19T13:05:00.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:05:01.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 807453851650 -lt 807453851650 2026-02-19T13:05:01.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:05:01.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:01.131 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:01.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:05:01.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:05:01.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:05:01.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:05:01.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:05:01.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:05:01.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:05:01.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:05:01.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:05:01.473 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:05:01.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:01.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:01.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:05:01.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:05:01.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:05:01.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2195: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ10 key2-ROBJ10 val2-ROBJ10 2026-02-19T13:05:01.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2196: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-19T13:05:01.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:05:01.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:05:01.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:05:01.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:05:01.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-19T13:05:01.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:05:01.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:05:01.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:05:01.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:05:01.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:05:01.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:05:01.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:05:01.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:05:01.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:05:01.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:05:01.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:05:01.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-19T13:05:01.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:05:01.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:05:01.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:05:01.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:05:01.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:05:01.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-19T13:05:02.172 INFO:tasks.workunit.client.0.trial144.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-19T13:05:02.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:05:02.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:05:02.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:05:02.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:05:02.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:05:02.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:05:02.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:05:02.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:05:02.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:05:02.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:05:02.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:05:02.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:05:02.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:05:02.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:05:02.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:05:02.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:05:02.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:05:02.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:05:02.275 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:05:02.275 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:02.275 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:02.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:05:02.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:05:02.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:05:02.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:05:02.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:05:02.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:05:02.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:05:02.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:05:02.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:05:02.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:05:02.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:05:02.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:05:02.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:05:02.278 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:05:02.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:05:02.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:05:02.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:05:02.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:05:02.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:05:02.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:05:02.292 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:02.290+0000 7fda6b157a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:02.294 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:02.290+0000 7fda6b157a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:02.295 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:02.294+0000 7fda6b157a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:02.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:05:02.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:05:02.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:05:02.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:05:02.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:05:02.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:05:02.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:05:02.439 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:05:02.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:05:02.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:05:02.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:05:02.720 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:02.718+0000 7fda6b157a40 -1 Falling back to public interface 2026-02-19T13:05:03.272 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:03.270+0000 7fda6b157a40 -1 osd.0 189 log_to_monitors true 2026-02-19T13:05:03.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:05:03.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:05:03.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:05:03.599 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:05:03.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:05:03.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:05:03.764 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 193 up_thru 193 down_at 190 last_clean_interval [183,189) [v2:127.0.0.1:6802/1177766083,v1:127.0.0.1:6803/1177766083] [v2:127.0.0.1:6804/1177766083,v1:127.0.0.1:6805/1177766083] exists,up 0250c601-08cf-4db7-8be3-271bfebc9558 2026-02-19T13:05:03.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:05:03.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:05:03.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:05:03.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:05:03.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:05:03.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:05:03.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:05:03.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:05:03.767 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:05:03.767 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:05:03.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:05:03.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:05:03.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:05:03.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:05:03.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:05:03.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:05:03.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:05:03.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:05:03.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:05:04.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:05:04.043 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:05:04.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:05:04.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:04.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:05:04.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=828928688130 2026-02-19T13:05:04.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 828928688130 2026-02-19T13:05:04.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-828928688130' 2026-02-19T13:05:04.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:04.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:05:04.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=807453851652 2026-02-19T13:05:04.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 807453851652 2026-02-19T13:05:04.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-828928688130 1-807453851652' 2026-02-19T13:05:04.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:04.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-828928688130 2026-02-19T13:05:04.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:04.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:05:04.194 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-828928688130 2026-02-19T13:05:04.194 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:04.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=828928688130 2026-02-19T13:05:04.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 828928688130' 2026-02-19T13:05:04.196 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 828928688130 2026-02-19T13:05:04.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:04.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 785979015172 -lt 828928688130 2026-02-19T13:05:04.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:05:05.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:05:05.369 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:05.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 828928688130 -lt 828928688130 2026-02-19T13:05:05.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:05.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-807453851652 2026-02-19T13:05:05.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:05.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:05:05.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-807453851652 2026-02-19T13:05:05.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:05.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=807453851652 2026-02-19T13:05:05.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 807453851652' 2026-02-19T13:05:05.531 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 807453851652 2026-02-19T13:05:05.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:05:05.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 807453851652 -lt 807453851652 2026-02-19T13:05:05.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:05:05.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:05.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:05.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:05:05.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:05:05.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:05:05.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:05:05.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:05:05.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:05:05.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:05:05.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:05:06.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:05:06.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:05:06.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:06.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:06.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:05:06.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:05:06.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:05:06.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2197: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-19T13:05:06.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:05:06.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:05:06.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:05:06.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:05:06.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-19T13:05:06.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:05:06.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:05:06.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:05:06.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:05:06.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:05:06.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:05:06.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:05:06.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:05:06.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:05:06.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:05:06.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:05:06.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-19T13:05:06.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:05:06.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:05:06.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:05:06.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:05:06.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:05:06.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-19T13:05:06.739 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-19T13:05:06.739 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-19T13:05:06.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:05:06.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:05:06.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:05:06.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:05:06.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:05:06.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:05:06.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:05:06.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:05:06.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:05:06.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:05:06.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:05:06.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:05:06.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:05:06.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:05:06.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:05:06.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:05:06.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:05:06.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:05:06.822 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:05:06.822 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:06.822 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:06.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:05:06.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:05:06.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:05:06.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:05:06.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:05:06.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:05:06.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:05:06.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:05:06.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:05:06.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:05:06.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:05:06.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:05:06.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:05:06.825 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:05:06.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:05:06.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:05:06.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:05:06.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:05:06.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:05:06.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:05:06.837 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:06.834+0000 7fd1105e5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:06.838 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:06.834+0000 7fd1105e5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:06.839 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:06.838+0000 7fd1105e5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:06.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:05:06.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:05:06.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:05:06.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:05:06.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:05:06.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:05:06.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:05:06.980 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:05:06.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:05:06.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:05:07.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:05:07.232 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:07.230+0000 7fd1105e5a40 -1 Falling back to public interface 2026-02-19T13:05:07.882 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:07.878+0000 7fd1105e5a40 -1 osd.1 195 log_to_monitors true 2026-02-19T13:05:08.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:05:08.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:05:08.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:05:08.143 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:05:08.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:05:08.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:05:08.304 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 199 up_thru 199 down_at 196 last_clean_interval [188,195) [v2:127.0.0.1:6810/1187635480,v1:127.0.0.1:6811/1187635480] [v2:127.0.0.1:6812/1187635480,v1:127.0.0.1:6813/1187635480] exists,up c7a4d5dd-9169-4e8d-826b-e9eca7704cb3 2026-02-19T13:05:08.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:05:08.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:05:08.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:05:08.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:05:08.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:05:08.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:05:08.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:05:08.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:05:08.306 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:05:08.306 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:05:08.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:05:08.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:05:08.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:05:08.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:05:08.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:05:08.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:05:08.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:05:08.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:05:08.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:05:08.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:05:08.589 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:05:08.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:05:08.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:08.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:05:08.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=828928688132 2026-02-19T13:05:08.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 828928688132 2026-02-19T13:05:08.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-828928688132' 2026-02-19T13:05:08.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:08.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:05:08.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=854698491906 2026-02-19T13:05:08.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 854698491906 2026-02-19T13:05:08.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-828928688132 1-854698491906' 2026-02-19T13:05:08.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:08.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-828928688132 2026-02-19T13:05:08.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:08.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:05:08.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-828928688132 2026-02-19T13:05:08.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:08.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=828928688132 2026-02-19T13:05:08.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 828928688132' 2026-02-19T13:05:08.742 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 828928688132 2026-02-19T13:05:08.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:08.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 828928688131 -lt 828928688132 2026-02-19T13:05:08.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:05:09.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:05:09.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:10.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 828928688131 -lt 828928688132 2026-02-19T13:05:10.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:05:11.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:05:11.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:11.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 828928688132 -lt 828928688132 2026-02-19T13:05:11.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:11.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-854698491906 2026-02-19T13:05:11.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:11.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:05:11.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-854698491906 2026-02-19T13:05:11.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:11.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=854698491906 2026-02-19T13:05:11.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 854698491906' 2026-02-19T13:05:11.234 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 854698491906 2026-02-19T13:05:11.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:05:11.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 854698491906 -lt 854698491906 2026-02-19T13:05:11.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:05:11.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:11.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:11.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:05:11.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:05:11.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:05:11.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:05:11.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:05:11.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:05:11.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:05:11.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:05:11.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:05:11.758 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:05:11.758 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:11.758 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:11.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:05:11.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:05:11.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:05:11.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2198: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/oi 2026-02-19T13:05:11.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2200: TEST_corrupt_scrub_replicated: inject_eio rep data csr_pool ROBJ11 td/osd-scrub-repair 0 2026-02-19T13:05:11.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=rep 2026-02-19T13:05:11.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:05:11.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:05:11.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:05:11.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=csr_pool 2026-02-19T13:05:11.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:05:11.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=ROBJ11 2026-02-19T13:05:11.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:05:11.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/osd-scrub-repair 2026-02-19T13:05:11.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:05:11.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-19T13:05:11.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:05:11.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds csr_pool ROBJ11 2026-02-19T13:05:11.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=csr_pool 2026-02-19T13:05:11.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=ROBJ11 2026-02-19T13:05:11.956 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map csr_pool ROBJ11 2026-02-19T13:05:11.956 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:05:12.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-19T13:05:12.115 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:05:12.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-19T13:05:12.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('1' '0') 2026-02-19T13:05:12.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:05:12.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-19T13:05:12.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' rep '!=' ec ']' 2026-02-19T13:05:12.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2497: inject_eio: shard_id= 2026-02-19T13:05:12.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/osd-scrub-repair/1/type 2026-02-19T13:05:12.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:05:12.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T13:05:12.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:05:12.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T13:05:12.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:05:12.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:05:12.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:05:12.119 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T13:05:12.119 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:05:12.119 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:05:12.119 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:05:12.119 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:12.119 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:12.119 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T13:05:12.120 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:05:12.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:05:12.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:05:12.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:05:12.183 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-19T13:05:12.183 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:05:12.183 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:05:12.183 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:05:12.183 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:12.183 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:12.183 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T13:05:12.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:05:12.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.35818/ceph-osd.1.asok injectdataerr csr_pool ROBJ11 2026-02-19T13:05:12.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2201: TEST_corrupt_scrub_replicated: inject_eio rep mdata csr_pool ROBJ12 td/osd-scrub-repair 1 2026-02-19T13:05:12.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=rep 2026-02-19T13:05:12.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:05:12.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=mdata 2026-02-19T13:05:12.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:05:12.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=csr_pool 2026-02-19T13:05:12.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:05:12.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=ROBJ12 2026-02-19T13:05:12.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:05:12.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/osd-scrub-repair 2026-02-19T13:05:12.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:05:12.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-19T13:05:12.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:05:12.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds csr_pool ROBJ12 2026-02-19T13:05:12.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=csr_pool 2026-02-19T13:05:12.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=ROBJ12 2026-02-19T13:05:12.246 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map csr_pool ROBJ12 2026-02-19T13:05:12.246 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:05:12.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-19T13:05:12.400 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:05:12.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-19T13:05:12.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('1' '0') 2026-02-19T13:05:12.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:05:12.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=0 2026-02-19T13:05:12.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' rep '!=' ec ']' 2026-02-19T13:05:12.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2497: inject_eio: shard_id= 2026-02-19T13:05:12.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/osd-scrub-repair/0/type 2026-02-19T13:05:12.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:05:12.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-19T13:05:12.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:05:12.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=0 2026-02-19T13:05:12.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:05:12.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:05:12.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:05:12.404 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.0 2026-02-19T13:05:12.404 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:05:12.404 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:05:12.404 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:05:12.404 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:12.404 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:12.404 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.0.asok 2026-02-19T13:05:12.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:05:12.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:05:12.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:05:12.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:05:12.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.0 2026-02-19T13:05:12.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:05:12.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:05:12.465 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:05:12.465 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:12.465 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:12.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.0.asok 2026-02-19T13:05:12.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:05:12.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.35818/ceph-osd.0.asok injectmdataerr csr_pool ROBJ12 2026-02-19T13:05:12.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2202: TEST_corrupt_scrub_replicated: inject_eio rep mdata csr_pool ROBJ13 td/osd-scrub-repair 1 2026-02-19T13:05:12.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=rep 2026-02-19T13:05:12.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:05:12.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=mdata 2026-02-19T13:05:12.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:05:12.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=csr_pool 2026-02-19T13:05:12.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:05:12.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=ROBJ13 2026-02-19T13:05:12.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:05:12.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/osd-scrub-repair 2026-02-19T13:05:12.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:05:12.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-19T13:05:12.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:05:12.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds csr_pool ROBJ13 2026-02-19T13:05:12.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=csr_pool 2026-02-19T13:05:12.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=ROBJ13 2026-02-19T13:05:12.527 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map csr_pool ROBJ13 2026-02-19T13:05:12.527 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:05:12.688 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-19T13:05:12.688 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:05:12.688 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-19T13:05:12.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('1' '0') 2026-02-19T13:05:12.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:05:12.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=0 2026-02-19T13:05:12.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' rep '!=' ec ']' 2026-02-19T13:05:12.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2497: inject_eio: shard_id= 2026-02-19T13:05:12.689 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/osd-scrub-repair/0/type 2026-02-19T13:05:12.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:05:12.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-19T13:05:12.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:05:12.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=0 2026-02-19T13:05:12.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:05:12.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:05:12.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:05:12.692 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.0 2026-02-19T13:05:12.692 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:05:12.692 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:05:12.692 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:05:12.692 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:12.692 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:12.692 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.0.asok 2026-02-19T13:05:12.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:05:12.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:05:12.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:05:12.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:05:12.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.0 2026-02-19T13:05:12.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:05:12.752 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:05:12.752 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:05:12.753 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:12.753 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:12.753 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.0.asok 2026-02-19T13:05:12.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:05:12.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.35818/ceph-osd.0.asok injectmdataerr csr_pool ROBJ13 2026-02-19T13:05:12.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2203: TEST_corrupt_scrub_replicated: inject_eio rep data csr_pool ROBJ13 td/osd-scrub-repair 0 2026-02-19T13:05:12.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=rep 2026-02-19T13:05:12.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:05:12.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:05:12.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:05:12.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=csr_pool 2026-02-19T13:05:12.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:05:12.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=ROBJ13 2026-02-19T13:05:12.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:05:12.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/osd-scrub-repair 2026-02-19T13:05:12.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:05:12.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-19T13:05:12.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:05:12.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds csr_pool ROBJ13 2026-02-19T13:05:12.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=csr_pool 2026-02-19T13:05:12.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=ROBJ13 2026-02-19T13:05:12.814 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map csr_pool ROBJ13 2026-02-19T13:05:12.814 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:05:12.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-19T13:05:12.973 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:05:12.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-19T13:05:12.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('1' '0') 2026-02-19T13:05:12.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:05:12.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-19T13:05:12.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' rep '!=' ec ']' 2026-02-19T13:05:12.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2497: inject_eio: shard_id= 2026-02-19T13:05:12.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/osd-scrub-repair/1/type 2026-02-19T13:05:12.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:05:12.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T13:05:12.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:05:12.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T13:05:12.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:05:12.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:05:12.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:05:12.977 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T13:05:12.977 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:05:12.977 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:05:12.977 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:05:12.977 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:12.977 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:12.978 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T13:05:12.978 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:05:13.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:05:13.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:05:13.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:05:13.039 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-19T13:05:13.039 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:05:13.039 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:05:13.039 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:05:13.039 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:13.039 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:13.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T13:05:13.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:05:13.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.35818/ceph-osd.1.asok injectdataerr csr_pool ROBJ13 2026-02-19T13:05:13.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2206: TEST_corrupt_scrub_replicated: ceph tell 'osd.*' injectargs -- --osd-max-object-size=134217728 2026-02-19T13:05:13.164 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: {} 2026-02-19T13:05:13.164 INFO:tasks.workunit.client.0.trial144.stderr:osd.0: osd_max_object_size = '' 2026-02-19T13:05:13.172 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: {} 2026-02-19T13:05:13.173 INFO:tasks.workunit.client.0.trial144.stderr:osd.1: osd_max_object_size = '' 2026-02-19T13:05:13.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2208: TEST_corrupt_scrub_replicated: pg_deep_scrub 3.0 2026-02-19T13:05:13.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_deep_scrub: local pgid=3.0 2026-02-19T13:05:13.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_deep_scrub: wait_for_pg_clean 3.0 2026-02-19T13:05:13.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=3.0 2026-02-19T13:05:13.183 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:05:13.184 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:05:13.184 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:05:13.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:05:13.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:05:13.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:05:13.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:05:13.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:05:13.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:05:13.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:05:13.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:05:13.485 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:05:13.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:05:13.642 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:05:13.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:05:13.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:13.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:05:13.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=828928688134 2026-02-19T13:05:13.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 828928688134 2026-02-19T13:05:13.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-828928688134' 2026-02-19T13:05:13.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:13.714 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:05:13.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=854698491908 2026-02-19T13:05:13.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 854698491908 2026-02-19T13:05:13.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-828928688134 1-854698491908' 2026-02-19T13:05:13.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:13.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-828928688134 2026-02-19T13:05:13.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:13.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:05:13.791 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-828928688134 2026-02-19T13:05:13.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:13.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=828928688134 2026-02-19T13:05:13.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 828928688134' 2026-02-19T13:05:13.794 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 828928688134 2026-02-19T13:05:13.794 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:13.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 828928688132 -lt 828928688134 2026-02-19T13:05:13.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:05:14.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:05:14.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:15.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 828928688134 -lt 828928688134 2026-02-19T13:05:15.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:15.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-854698491908 2026-02-19T13:05:15.126 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:15.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:05:15.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-854698491908 2026-02-19T13:05:15.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:15.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=854698491908 2026-02-19T13:05:15.130 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 854698491908 2026-02-19T13:05:15.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 854698491908' 2026-02-19T13:05:15.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:05:15.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 854698491908 -lt 854698491908 2026-02-19T13:05:15.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:05:15.293 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 3.0 loop 0 2026-02-19T13:05:15.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-19T13:05:15.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 3.0 2026-02-19T13:05:15.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=3.0 2026-02-19T13:05:15.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:05:15.294 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 3.0 query 2026-02-19T13:05:15.294 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:05:15.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-19T13:05:15.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:05:15.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:05:15.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:05:15.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:05:15.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:05:15.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:05:15.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:05:15.369 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:05:15.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: local last_scrub=2026-02-19T13:02:55.699858+0000 2026-02-19T13:05:15.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_deep_scrub: ceph pg deep-scrub 3.0 2026-02-19T13:05:15.671 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2026-02-19T13:05:15.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_deep_scrub: wait_for_scrub 3.0 2026-02-19T13:02:55.699858+0000 last_deep_scrub_stamp 2026-02-19T13:05:15.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:05:15.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:02:55.699858+0000 2026-02-19T13:05:15.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-19T13:05:15.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:05:15.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:05:15.682 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:05:15.682 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:05:15.682 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:05:15.682 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:05:15.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:05:15.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:02:55.699858+0000 '>' 2026-02-19T13:02:55.699858+0000 2026-02-19T13:05:15.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:05:16.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:05:16.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:05:16.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:05:16.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:05:16.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:05:16.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:05:16.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:05:17.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:02:55.699858+0000 '>' 2026-02-19T13:02:55.699858+0000 2026-02-19T13:05:17.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:05:18.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:05:18.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:05:18.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:05:18.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:05:18.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:05:18.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:05:18.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:05:18.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:02:55.699858+0000 '>' 2026-02-19T13:02:55.699858+0000 2026-02-19T13:05:18.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:05:19.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:05:19.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:05:19.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:05:19.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:05:19.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:05:19.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:05:19.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:05:15.818742+0000 '>' 2026-02-19T13:02:55.699858+0000 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2210: TEST_corrupt_scrub_replicated: err_strings=() 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2211: TEST_corrupt_scrub_replicated: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:30259878:::ROBJ15:head : candidate had a missing info key' 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2212: TEST_corrupt_scrub_replicated: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:33aca486:::ROBJ18:head : data_digest 0xbd89c912 != data_digest 0x2ddbf8f5 from auth oi 3:33aca486:::ROBJ18:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 54 dd 2ddbf8f5 od ddc3680f alloc_hint [[]0 0 255[]][)], object info inconsistent ' 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2213: TEST_corrupt_scrub_replicated: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:33aca486:::ROBJ18:head : data_digest 0xbd89c912 != data_digest 0x2ddbf8f5 from auth oi 3:33aca486:::ROBJ18:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 54 dd 2ddbf8f5 od ddc3680f alloc_hint [[]0 0 255[]][)]' 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2214: TEST_corrupt_scrub_replicated: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:33aca486:::ROBJ18:head : failed to pick suitable auth object' 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2215: TEST_corrupt_scrub_replicated: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:5c7b2c47:::ROBJ16:head : candidate had a corrupt snapset' 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2216: TEST_corrupt_scrub_replicated: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:5c7b2c47:::ROBJ16:head : candidate had a missing snapset key' 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2217: TEST_corrupt_scrub_replicated: err_strings[6]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:5c7b2c47:::ROBJ16:head : failed to pick suitable object info' 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2218: TEST_corrupt_scrub_replicated: err_strings[7]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:86586531:::ROBJ8:head : attr value mismatch '\''_key1-ROBJ8'\'', attr name mismatch '\''_key3-ROBJ8'\'', attr name mismatch '\''_key2-ROBJ8'\''' 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2219: TEST_corrupt_scrub_replicated: err_strings[8]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:87abbf36:::ROBJ11:head : candidate had a read error' 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2220: TEST_corrupt_scrub_replicated: err_strings[9]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:8aa5320e:::ROBJ17:head : data_digest 0x5af0c3ef != data_digest 0x2ddbf8f5 from auth oi 3:8aa5320e:::ROBJ17:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 51 dd 2ddbf8f5 od e9572720 alloc_hint [[]0 0 0[]][)]' 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2221: TEST_corrupt_scrub_replicated: err_strings[10]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:8aa5320e:::ROBJ17:head : data_digest 0x5af0c3ef != data_digest 0x2ddbf8f5 from auth oi 3:8aa5320e:::ROBJ17:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 51 dd 2ddbf8f5 od e9572720 alloc_hint [[]0 0 0[]][)]' 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2222: TEST_corrupt_scrub_replicated: err_strings[11]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:8aa5320e:::ROBJ17:head : failed to pick suitable auth object' 2026-02-19T13:05:19.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2223: TEST_corrupt_scrub_replicated: err_strings[12]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:8b55fa4b:::ROBJ7:head : omap_digest 0xefced57a != omap_digest 0x6a73cc07 from shard 1' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2224: TEST_corrupt_scrub_replicated: err_strings[13]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:8b55fa4b:::ROBJ7:head : omap_digest 0x6a73cc07 != omap_digest 0xefced57a from auth oi 3:8b55fa4b:::ROBJ7:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 21 dd 2ddbf8f5 od efced57a alloc_hint [[]0 0 0[]][)]' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2225: TEST_corrupt_scrub_replicated: err_strings[14]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:a53c12e8:::ROBJ6:head : omap_digest 0x689ee887 != omap_digest 0x179c919f from shard 1, omap_digest 0x689ee887 != omap_digest 0x179c919f from auth oi 3:a53c12e8:::ROBJ6:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 18 dd 2ddbf8f5 od 179c919f alloc_hint [[]0 0 0[]][)]' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2226: TEST_corrupt_scrub_replicated: err_strings[15]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:b1f19cbd:::ROBJ10:head : omap_digest 0xa8dd5adc != omap_digest 0xc2025a24 from auth oi 3:b1f19cbd:::ROBJ10:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 30 dd 2ddbf8f5 od c2025a24 alloc_hint [[]0 0 0[]][)]' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2227: TEST_corrupt_scrub_replicated: err_strings[16]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:b1f19cbd:::ROBJ10:head : omap_digest 0xa8dd5adc != omap_digest 0xc2025a24 from auth oi 3:b1f19cbd:::ROBJ10:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 30 dd 2ddbf8f5 od c2025a24 alloc_hint [[]0 0 0[]][)]' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2228: TEST_corrupt_scrub_replicated: err_strings[17]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:b1f19cbd:::ROBJ10:head : failed to pick suitable auth object' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2229: TEST_corrupt_scrub_replicated: err_strings[18]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:bc819597:::ROBJ12:head : candidate had a stat error' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2230: TEST_corrupt_scrub_replicated: err_strings[19]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:c0c86b1d:::ROBJ14:head : candidate had a missing info key' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2231: TEST_corrupt_scrub_replicated: err_strings[20]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:c0c86b1d:::ROBJ14:head : candidate had a corrupt info' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2232: TEST_corrupt_scrub_replicated: err_strings[21]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:c0c86b1d:::ROBJ14:head : failed to pick suitable object info' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2233: TEST_corrupt_scrub_replicated: err_strings[22]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : candidate size 9 info size 7 mismatch' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2234: TEST_corrupt_scrub_replicated: err_strings[23]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : data_digest 0x2d4a11c2 != data_digest 0x2ddbf8f5 from shard 0, data_digest 0x2d4a11c2 != data_digest 0x2ddbf8f5 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from shard 0' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2235: TEST_corrupt_scrub_replicated: err_strings[24]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:d60617f9:::ROBJ13:head : candidate had a read error' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2236: TEST_corrupt_scrub_replicated: err_strings[25]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:d60617f9:::ROBJ13:head : candidate had a stat error' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2237: TEST_corrupt_scrub_replicated: err_strings[26]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:d60617f9:::ROBJ13:head : failed to pick suitable object info' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2238: TEST_corrupt_scrub_replicated: err_strings[27]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:e97ce31e:::ROBJ2:head : data_digest 0x578a4830 != data_digest 0x2ddbf8f5 from shard 1, data_digest 0x578a4830 != data_digest 0x2ddbf8f5 from auth oi 3:e97ce31e:::ROBJ2:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 6 dd 2ddbf8f5 od f8e11918 alloc_hint [[]0 0 0[]][)]' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2239: TEST_corrupt_scrub_replicated: err_strings[28]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 3:f2a5b2a4:::ROBJ3:head : missing' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2240: TEST_corrupt_scrub_replicated: err_strings[29]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:f4981d31:::ROBJ4:head : omap_digest 0xd7178dfe != omap_digest 0xe2d46ea4 from shard 1, omap_digest 0xd7178dfe != omap_digest 0xe2d46ea4 from auth oi 3:f4981d31:::ROBJ4:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 12 dd 2ddbf8f5 od e2d46ea4 alloc_hint [[]0 0 0[]][)]' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2241: TEST_corrupt_scrub_replicated: err_strings[30]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:f4bfd4d1:::ROBJ5:head : omap_digest 0x1a862a41 != omap_digest 0x6cac8f6 from shard 1' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2242: TEST_corrupt_scrub_replicated: err_strings[31]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:f4bfd4d1:::ROBJ5:head : omap_digest 0x6cac8f6 != omap_digest 0x1a862a41 from auth oi 3:f4bfd4d1:::ROBJ5:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 15 dd 2ddbf8f5 od 1a862a41 alloc_hint [[]0 0 0[]][)]' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2243: TEST_corrupt_scrub_replicated: err_strings[32]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:ffdb2004:::ROBJ9:head : candidate size 3 info size 7 mismatch' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2244: TEST_corrupt_scrub_replicated: err_strings[33]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:ffdb2004:::ROBJ9:head : object info inconsistent ' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2245: TEST_corrupt_scrub_replicated: err_strings[34]='log_channel[(]cluster[)] log [[]ERR[]] : deep-scrub [0-9]*[.]0 3:c0c86b1d:::ROBJ14:head : no '\''_'\'' attr' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2246: TEST_corrupt_scrub_replicated: err_strings[35]='log_channel[(]cluster[)] log [[]ERR[]] : deep-scrub [0-9]*[.]0 3:5c7b2c47:::ROBJ16:head : can'\''t decode '\''snapset'\'' attr .* v=3 cannot decode .* Malformed input' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2247: TEST_corrupt_scrub_replicated: err_strings[36]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub : stat mismatch, got 19/19 objects, 0/0 clones, 18/19 dirty, 18/19 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 1049715/1049716 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2248: TEST_corrupt_scrub_replicated: err_strings[37]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub 1 missing, 11 inconsistent objects' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2249: TEST_corrupt_scrub_replicated: err_strings[38]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub 35 errors' 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:30259878:::ROBJ15:head : candidate had a missing info key' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:33aca486:::ROBJ18:head : data_digest 0xbd89c912 != data_digest 0x2ddbf8f5 from auth oi 3:33aca486:::ROBJ18:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 54 dd 2ddbf8f5 od ddc3680f alloc_hint [[]0 0 255[]][)], object info inconsistent ' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:33aca486:::ROBJ18:head : data_digest 0xbd89c912 != data_digest 0x2ddbf8f5 from auth oi 3:33aca486:::ROBJ18:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 54 dd 2ddbf8f5 od ddc3680f alloc_hint [[]0 0 255[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:33aca486:::ROBJ18:head : failed to pick suitable auth object' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:5c7b2c47:::ROBJ16:head : candidate had a corrupt snapset' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:5c7b2c47:::ROBJ16:head : candidate had a missing snapset key' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:5c7b2c47:::ROBJ16:head : failed to pick suitable object info' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:86586531:::ROBJ8:head : attr value mismatch '\''_key1-ROBJ8'\'', attr name mismatch '\''_key3-ROBJ8'\'', attr name mismatch '\''_key2-ROBJ8'\''' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:87abbf36:::ROBJ11:head : candidate had a read error' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:8aa5320e:::ROBJ17:head : data_digest 0x5af0c3ef != data_digest 0x2ddbf8f5 from auth oi 3:8aa5320e:::ROBJ17:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 51 dd 2ddbf8f5 od e9572720 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:8aa5320e:::ROBJ17:head : data_digest 0x5af0c3ef != data_digest 0x2ddbf8f5 from auth oi 3:8aa5320e:::ROBJ17:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 51 dd 2ddbf8f5 od e9572720 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:8aa5320e:::ROBJ17:head : failed to pick suitable auth object' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:8b55fa4b:::ROBJ7:head : omap_digest 0xefced57a != omap_digest 0x6a73cc07 from shard 1' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:8b55fa4b:::ROBJ7:head : omap_digest 0x6a73cc07 != omap_digest 0xefced57a from auth oi 3:8b55fa4b:::ROBJ7:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 21 dd 2ddbf8f5 od efced57a alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:a53c12e8:::ROBJ6:head : omap_digest 0x689ee887 != omap_digest 0x179c919f from shard 1, omap_digest 0x689ee887 != omap_digest 0x179c919f from auth oi 3:a53c12e8:::ROBJ6:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 18 dd 2ddbf8f5 od 179c919f alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:b1f19cbd:::ROBJ10:head : omap_digest 0xa8dd5adc != omap_digest 0xc2025a24 from auth oi 3:b1f19cbd:::ROBJ10:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 30 dd 2ddbf8f5 od c2025a24 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:b1f19cbd:::ROBJ10:head : omap_digest 0xa8dd5adc != omap_digest 0xc2025a24 from auth oi 3:b1f19cbd:::ROBJ10:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 30 dd 2ddbf8f5 od c2025a24 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:b1f19cbd:::ROBJ10:head : failed to pick suitable auth object' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:bc819597:::ROBJ12:head : candidate had a stat error' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:c0c86b1d:::ROBJ14:head : candidate had a missing info key' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:c0c86b1d:::ROBJ14:head : candidate had a corrupt info' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:c0c86b1d:::ROBJ14:head : failed to pick suitable object info' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : candidate size 9 info size 7 mismatch' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : data_digest 0x2d4a11c2 != data_digest 0x2ddbf8f5 from shard 0, data_digest 0x2d4a11c2 != data_digest 0x2ddbf8f5 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from shard 0' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:d60617f9:::ROBJ13:head : candidate had a read error' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:d60617f9:::ROBJ13:head : candidate had a stat error' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:d60617f9:::ROBJ13:head : failed to pick suitable object info' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:e97ce31e:::ROBJ2:head : data_digest 0x578a4830 != data_digest 0x2ddbf8f5 from shard 1, data_digest 0x578a4830 != data_digest 0x2ddbf8f5 from auth oi 3:e97ce31e:::ROBJ2:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 6 dd 2ddbf8f5 od f8e11918 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 3:f2a5b2a4:::ROBJ3:head : missing' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:f4981d31:::ROBJ4:head : omap_digest 0xd7178dfe != omap_digest 0xe2d46ea4 from shard 1, omap_digest 0xd7178dfe != omap_digest 0xe2d46ea4 from auth oi 3:f4981d31:::ROBJ4:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 12 dd 2ddbf8f5 od e2d46ea4 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:f4bfd4d1:::ROBJ5:head : omap_digest 0x1a862a41 != omap_digest 0x6cac8f6 from shard 1' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:f4bfd4d1:::ROBJ5:head : omap_digest 0x6cac8f6 != omap_digest 0x1a862a41 from auth oi 3:f4bfd4d1:::ROBJ5:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 15 dd 2ddbf8f5 od 1a862a41 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:ffdb2004:::ROBJ9:head : candidate size 3 info size 7 mismatch' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:ffdb2004:::ROBJ9:head : object info inconsistent ' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : deep-scrub [0-9]*[.]0 3:c0c86b1d:::ROBJ14:head : no '\''_'\'' attr' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : deep-scrub [0-9]*[.]0 3:5c7b2c47:::ROBJ16:head : can'\''t decode '\''snapset'\'' attr .* v=3 cannot decode .* Malformed input' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub : stat mismatch, got 19/19 objects, 0/0 clones, 18/19 dirty, 18/19 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 1049715/1049716 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub 1 missing, 11 inconsistent objects' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2251: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-19T13:05:19.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2253: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub 35 errors' td/osd-scrub-repair/osd.1.log 2026-02-19T13:05:19.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2260: TEST_corrupt_scrub_replicated: rados list-inconsistent-pg csr_pool 2026-02-19T13:05:19.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2262: TEST_corrupt_scrub_replicated: jq '. | length' td/osd-scrub-repair/json 2026-02-19T13:05:19.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2262: TEST_corrupt_scrub_replicated: test 1 = 1 2026-02-19T13:05:19.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2264: TEST_corrupt_scrub_replicated: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-19T13:05:19.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2264: TEST_corrupt_scrub_replicated: test 3.0 = 3.0 2026-02-19T13:05:19.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2266: TEST_corrupt_scrub_replicated: rados list-inconsistent-obj 3.0 2026-02-19T13:05:19.702 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2268: TEST_corrupt_scrub_replicated: jq .epoch td/osd-scrub-repair/json 2026-02-19T13:05:19.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2268: TEST_corrupt_scrub_replicated: epoch=199 2026-02-19T13:05:19.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2270: TEST_corrupt_scrub_replicated: jq 'def walk(f): 2026-02-19T13:05:19.712 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:05:19.712 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:05:19.712 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:05:19.712 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:05:19.713 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:05:19.713 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:05:19.713 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:05:19.713 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:05:19.713 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:05:19.713 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:05:19.713 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:05:19.713 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-19T13:05:19.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2270: TEST_corrupt_scrub_replicated: jq .inconsistents 2026-02-19T13:05:19.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2270: TEST_corrupt_scrub_replicated: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:05:19.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3698: TEST_corrupt_scrub_replicated: jq 'def walk(f): 2026-02-19T13:05:19.736 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:05:19.736 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:05:19.736 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:05:19.737 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:05:19.737 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:05:19.737 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:05:19.737 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:05:19.737 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:05:19.737 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:05:19.737 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:05:19.737 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:05:19.737 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-19T13:05:19.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3698: TEST_corrupt_scrub_replicated: jq .inconsistents 2026-02-19T13:05:19.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3698: TEST_corrupt_scrub_replicated: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:05:19.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3699: TEST_corrupt_scrub_replicated: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-19T13:05:19.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-19T13:05:19.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3700: TEST_corrupt_scrub_replicated: test no = yes 2026-02-19T13:05:19.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3705: TEST_corrupt_scrub_replicated: test '' = yes 2026-02-19T13:05:19.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3710: TEST_corrupt_scrub_replicated: repair 3.0 2026-02-19T13:05:19.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=3.0 2026-02-19T13:05:19.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 3.0 2026-02-19T13:05:19.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:05:19.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:05:19.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:05:19.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:05:19.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T13:05:15.818742+0000 2026-02-19T13:05:19.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 3.0 2026-02-19T13:05:20.071 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0 on osd.1 to repair 2026-02-19T13:05:20.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 3.0 2026-02-19T13:05:15.818742+0000 2026-02-19T13:05:20.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:05:20.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:05:15.818742+0000 2026-02-19T13:05:20.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:05:20.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:05:20.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:05:20.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:05:20.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:05:20.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:05:20.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:05:20.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:05:20.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:05:15.818742+0000 '>' 2026-02-19T13:05:15.818742+0000 2026-02-19T13:05:20.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:05:21.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:05:21.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:05:21.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:05:21.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:05:21.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:05:21.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:05:21.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:05:21.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:05:15.818742+0000 '>' 2026-02-19T13:05:15.818742+0000 2026-02-19T13:05:21.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:05:22.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:05:22.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:05:22.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:05:22.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:05:22.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:05:22.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:05:22.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:05:22.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:05:15.818742+0000 '>' 2026-02-19T13:05:15.818742+0000 2026-02-19T13:05:22.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:05:23.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:05:23.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:05:23.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:05:23.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:05:23.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:05:23.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:05:23.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:05:23.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:05:15.818742+0000 '>' 2026-02-19T13:05:15.818742+0000 2026-02-19T13:05:23.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:05:24.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:05:24.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:05:24.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:05:24.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:05:24.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:05:24.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:05:24.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:05:24.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:05:21.793756+0000 '>' 2026-02-19T13:05:15.818742+0000 2026-02-19T13:05:24.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:05:24.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3711: TEST_corrupt_scrub_replicated: wait_for_clean 2026-02-19T13:05:24.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:05:24.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:05:24.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:05:24.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:05:24.887 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:05:24.887 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:05:24.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:05:24.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:05:24.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:05:25.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:05:25.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:05:25.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:05:25.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:05:25.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:05:25.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:05:25.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:05:25.162 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:05:25.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:05:25.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:25.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:05:25.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=828928688137 2026-02-19T13:05:25.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 828928688137 2026-02-19T13:05:25.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-828928688137' 2026-02-19T13:05:25.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:25.236 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:05:25.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=854698491911 2026-02-19T13:05:25.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 854698491911 2026-02-19T13:05:25.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-828928688137 1-854698491911' 2026-02-19T13:05:25.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:25.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-828928688137 2026-02-19T13:05:25.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:25.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:05:25.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-828928688137 2026-02-19T13:05:25.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:25.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=828928688137 2026-02-19T13:05:25.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 828928688137' 2026-02-19T13:05:25.314 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 828928688137 2026-02-19T13:05:25.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:25.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 828928688136 -lt 828928688137 2026-02-19T13:05:25.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:05:26.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:05:26.473 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:26.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 828928688137 -lt 828928688137 2026-02-19T13:05:26.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:26.636 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-854698491911 2026-02-19T13:05:26.636 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:26.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:05:26.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-854698491911 2026-02-19T13:05:26.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:26.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=854698491911 2026-02-19T13:05:26.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 854698491911' 2026-02-19T13:05:26.641 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 854698491911 2026-02-19T13:05:26.641 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:05:26.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 854698491911 -lt 854698491911 2026-02-19T13:05:26.811 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:05:26.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:26.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:27.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:05:27.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:05:27.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:05:27.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:05:27.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:05:27.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:05:27.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:05:27.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:05:27.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:05:27.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:05:27.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:27.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:27.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:05:27.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:05:27.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:05:27.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3714: TEST_corrupt_scrub_replicated: timeout 30 rados -p csr_pool get ROBJ17 td/osd-scrub-repair/robj17.out 2026-02-19T13:05:27.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3715: TEST_corrupt_scrub_replicated: timeout 30 rados -p csr_pool get ROBJ18 td/osd-scrub-repair/robj18.out 2026-02-19T13:05:27.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3717: TEST_corrupt_scrub_replicated: diff -q td/osd-scrub-repair/new.ROBJ17 td/osd-scrub-repair/robj17.out 2026-02-19T13:05:27.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3718: TEST_corrupt_scrub_replicated: rm -f td/osd-scrub-repair/new.ROBJ17 td/osd-scrub-repair/robj17.out 2026-02-19T13:05:27.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3719: TEST_corrupt_scrub_replicated: diff -q td/osd-scrub-repair/new.ROBJ18 td/osd-scrub-repair/robj18.out 2026-02-19T13:05:27.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3720: TEST_corrupt_scrub_replicated: rm -f td/osd-scrub-repair/new.ROBJ18 td/osd-scrub-repair/robj18.out 2026-02-19T13:05:27.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3722: TEST_corrupt_scrub_replicated: '[' 0 '!=' 0 ']' 2026-02-19T13:05:27.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3728: TEST_corrupt_scrub_replicated: ceph osd pool rm csr_pool csr_pool --yes-i-really-really-mean-it 2026-02-19T13:05:27.610 INFO:tasks.workunit.client.0.trial144.stderr:pool 'csr_pool' removed 2026-02-19T13:05:27.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T13:05:27.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:05:27.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:05:27.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:05:27.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:05:27.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:05:27.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:05:27.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:05:27.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:05:27.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:05:27.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:05:27.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:05:27.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:05:27.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:05:27.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:05:27.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:05:27.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:05:27.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:05:27.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:05:27.943 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:05:27.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:05:27.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:05:27.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:05:27.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:05:27.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:27.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:27.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:05:27.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:05:27.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:05:27.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T13:05:27.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T13:05:27.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T13:05:27.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T13:05:27.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:05:27.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:05:27.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:05:27.985 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:05:27.985 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:05:27.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:05:27.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:05:27.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:05:27.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:05:27.987 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:05:27.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:05:27.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:05:27.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:05:27.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:05:27.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:05:27.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:05:27.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:05:27.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:05:27.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:05:27.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:05:27.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:05:27.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:05:27.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:05:27.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:27.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:27.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:05:27.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:05:27.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:05:27.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T13:05:28.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:05:28.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:28.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:28.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T13:05:28.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:05:28.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:05:28.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:05:28.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T13:05:28.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_snapset_scrub_rep td/osd-scrub-repair 2026-02-19T13:05:28.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6100: TEST_corrupt_snapset_scrub_rep: local dir=td/osd-scrub-repair 2026-02-19T13:05:28.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6101: TEST_corrupt_snapset_scrub_rep: local poolname=csr_pool 2026-02-19T13:05:28.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6102: TEST_corrupt_snapset_scrub_rep: local total_objs=2 2026-02-19T13:05:28.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6104: TEST_corrupt_snapset_scrub_rep: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-19T13:05:28.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T13:05:28.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:05:28.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:05:28.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:05:28.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T13:05:28.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-19T13:05:28.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:05:28.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:05:28.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:05:28.026 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:05:28.026 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:28.026 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:28.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:05:28.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-19T13:05:28.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:05:28.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:05:28.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:05:28.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:05:28.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:05:28.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:05:28.058 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:05:28.058 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:05:28.058 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:05:28.059 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:05:28.059 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:28.059 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:28.060 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:05:28.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:05:28.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T13:05:28.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:05:28.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:05:28.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:05:28.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:05:28.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:05:28.117 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:05:28.117 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:05:28.117 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:05:28.117 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:05:28.117 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:28.117 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:28.118 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:05:28.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:05:28.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T13:05:28.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6105: TEST_corrupt_snapset_scrub_rep: run_mgr td/osd-scrub-repair x 2026-02-19T13:05:28.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T13:05:28.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:05:28.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:05:28.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:05:28.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T13:05:28.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:05:28.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:05:28.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:05:28.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:05:28.270 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:05:28.270 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:28.270 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:28.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:05:28.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:05:28.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:05:28.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6106: TEST_corrupt_snapset_scrub_rep: run_osd td/osd-scrub-repair 0 2026-02-19T13:05:28.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:05:28.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:05:28.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:05:28.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:05:28.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:05:28.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:05:28.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:05:28.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:05:28.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:05:28.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:05:28.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:05:28.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:05:28.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:05:28.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:05:28.293 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:05:28.293 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:05:28.293 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:05:28.293 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:05:28.293 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:28.293 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:28.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:05:28.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:05:28.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:05:28.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:05:28.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:05:28.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:05:28.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:05:28.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:05:28.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:05:28.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:05:28.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:05:28.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:05:28.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:05:28.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:05:28.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:05:28.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d342b0c5-b1ff-4f1a-a794-f47523dc01df 2026-02-19T13:05:28.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d342b0c5-b1ff-4f1a-a794-f47523dc01df' 2026-02-19T13:05:28.297 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 d342b0c5-b1ff-4f1a-a794-f47523dc01df 2026-02-19T13:05:28.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:05:28.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCYCpdp215nEhAAyKDK7+yW9jrxeZUmRuK2XA== 2026-02-19T13:05:28.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCYCpdp215nEhAAyKDK7+yW9jrxeZUmRuK2XA=="}' 2026-02-19T13:05:28.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d342b0c5-b1ff-4f1a-a794-f47523dc01df -i td/osd-scrub-repair/0/new.json 2026-02-19T13:05:28.397 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:05:28.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T13:05:28.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCYCpdp215nEhAAyKDK7+yW9jrxeZUmRuK2XA== --osd-uuid d342b0c5-b1ff-4f1a-a794-f47523dc01df 2026-02-19T13:05:28.422 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:28.418+0000 7f99826c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:28.423 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:28.422+0000 7f99826c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:28.424 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:28.422+0000 7f99826c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:28.425 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:28.422+0000 7f99826c8a40 -1 bdev(0x562d9805cc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:05:28.425 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:28.422+0000 7f99826c8a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T13:05:29.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T13:05:29.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:05:29.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:05:29.158 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:05:29.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:05:29.367 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:05:29.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:05:29.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:05:29.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:05:29.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:05:29.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:05:29.392 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:29.390+0000 7fa8578bea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:29.394 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:29.390+0000 7fa8578bea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:29.395 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:29.394+0000 7fa8578bea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:29.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:05:29.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:05:29.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:05:29.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:05:29.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:05:29.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:05:29.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:05:29.525 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:05:29.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:05:29.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:05:29.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:05:29.812 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:29.810+0000 7fa8578bea40 -1 Falling back to public interface 2026-02-19T13:05:30.368 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:30.366+0000 7fa8578bea40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:05:30.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:05:30.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:05:30.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:05:30.689 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:05:30.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:05:30.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:05:30.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:05:31.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:05:31.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:05:31.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:05:31.846 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:05:31.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:05:31.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:05:32.007 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2072233749,v1:127.0.0.1:6803/2072233749] [v2:127.0.0.1:6804/2072233749,v1:127.0.0.1:6805/2072233749] exists,up d342b0c5-b1ff-4f1a-a794-f47523dc01df 2026-02-19T13:05:32.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:05:32.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:05:32.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:05:32.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6107: TEST_corrupt_snapset_scrub_rep: run_osd td/osd-scrub-repair 1 2026-02-19T13:05:32.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:05:32.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:05:32.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:05:32.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:05:32.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:05:32.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:05:32.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:05:32.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:05:32.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:05:32.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:05:32.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:05:32.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:05:32.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:05:32.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:05:32.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:05:32.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:05:32.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:05:32.011 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:05:32.011 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:32.011 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:32.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:05:32.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:05:32.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:05:32.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:05:32.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:05:32.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:05:32.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:05:32.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:05:32.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:05:32.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:05:32.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:05:32.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:05:32.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:05:32.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:05:32.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:05:32.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0c152b5d-e631-47a6-bf8a-b2062d68495d 2026-02-19T13:05:32.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 0c152b5d-e631-47a6-bf8a-b2062d68495d' 2026-02-19T13:05:32.017 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 0c152b5d-e631-47a6-bf8a-b2062d68495d 2026-02-19T13:05:32.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:05:32.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCcCpdpWlGLARAAbe8aoWNeldj7+IiJS1GpuQ== 2026-02-19T13:05:32.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCcCpdpWlGLARAAbe8aoWNeldj7+IiJS1GpuQ=="}' 2026-02-19T13:05:32.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0c152b5d-e631-47a6-bf8a-b2062d68495d -i td/osd-scrub-repair/1/new.json 2026-02-19T13:05:32.189 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:05:32.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T13:05:32.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCcCpdpWlGLARAAbe8aoWNeldj7+IiJS1GpuQ== --osd-uuid 0c152b5d-e631-47a6-bf8a-b2062d68495d 2026-02-19T13:05:32.225 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:32.222+0000 7fa469a64a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:32.227 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:32.226+0000 7fa469a64a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:32.228 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:32.226+0000 7fa469a64a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:32.229 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:32.226+0000 7fa469a64a40 -1 bdev(0x562475f62c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:05:32.229 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:32.226+0000 7fa469a64a40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T13:05:32.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T13:05:32.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:05:32.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:05:32.891 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:05:32.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:05:33.095 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:05:33.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:05:33.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:05:33.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:05:33.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:05:33.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:05:33.120 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:33.118+0000 7f6646016a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:33.121 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:33.118+0000 7f6646016a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:33.123 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:33.122+0000 7f6646016a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:33.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:05:33.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:05:33.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:05:33.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:05:33.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:05:33.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:05:33.253 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:05:33.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:05:33.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:05:33.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:05:33.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:05:33.456 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:33.454+0000 7f6646016a40 -1 Falling back to public interface 2026-02-19T13:05:34.085 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:34.082+0000 7f6646016a40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:05:34.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:05:34.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:05:34.415 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:05:34.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:05:34.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:05:34.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:05:34.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:05:35.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:05:35.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:05:35.578 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:05:35.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:05:35.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:05:35.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:05:35.736 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3333306269,v1:127.0.0.1:6811/3333306269] [v2:127.0.0.1:6812/3333306269,v1:127.0.0.1:6813/3333306269] exists,up 0c152b5d-e631-47a6-bf8a-b2062d68495d 2026-02-19T13:05:35.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:05:35.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:05:35.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:05:35.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6108: TEST_corrupt_snapset_scrub_rep: create_rbd_pool 2026-02-19T13:05:35.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T13:05:35.883 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T13:05:35.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T13:05:35.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:05:36.102 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T13:05:36.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:05:37.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T13:05:37.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6109: TEST_corrupt_snapset_scrub_rep: wait_for_clean 2026-02-19T13:05:37.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:05:37.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:05:37.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:05:37.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:05:37.394 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:05:37.394 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:05:37.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:05:37.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:05:37.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:05:37.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:05:37.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:05:37.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:05:37.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:05:37.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:05:37.513 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:05:37.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:05:37.675 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:05:37.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:05:37.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:37.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:05:37.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T13:05:37.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T13:05:37.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T13:05:37.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:37.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:05:37.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672962 2026-02-19T13:05:37.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672962 2026-02-19T13:05:37.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-42949672962' 2026-02-19T13:05:37.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:37.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T13:05:37.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:37.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:05:37.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T13:05:37.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:37.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T13:05:37.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T13:05:37.827 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T13:05:37.827 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:37.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T13:05:37.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:05:38.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:05:38.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:39.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-19T13:05:39.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:39.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672962 2026-02-19T13:05:39.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:39.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:05:39.155 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672962 2026-02-19T13:05:39.155 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:39.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672962 2026-02-19T13:05:39.156 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672962 2026-02-19T13:05:39.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672962' 2026-02-19T13:05:39.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:05:39.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672962 -lt 42949672962 2026-02-19T13:05:39.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:05:39.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:39.318 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:39.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T13:05:39.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:05:39.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:05:39.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:05:39.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:05:39.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:05:39.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:05:39.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:05:39.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T13:05:39.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:05:39.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:39.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:39.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T13:05:39.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:05:39.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:05:39.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6111: TEST_corrupt_snapset_scrub_rep: create_pool foo 1 2026-02-19T13:05:39.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 1 2026-02-19T13:05:40.071 INFO:tasks.workunit.client.0.trial144.stderr:pool 'foo' created 2026-02-19T13:05:40.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:05:41.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6112: TEST_corrupt_snapset_scrub_rep: create_pool csr_pool 1 1 2026-02-19T13:05:41.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create csr_pool 1 1 2026-02-19T13:05:41.290 INFO:tasks.workunit.client.0.trial144.stderr:pool 'csr_pool' created 2026-02-19T13:05:41.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:05:42.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6113: TEST_corrupt_snapset_scrub_rep: wait_for_clean 2026-02-19T13:05:42.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:05:42.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:05:42.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:05:42.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:05:42.303 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:05:42.304 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:05:42.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:05:42.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:05:42.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:05:42.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:05:42.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:05:42.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:05:42.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:05:42.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:05:42.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:05:42.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:05:42.585 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:05:42.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:05:42.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:42.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:05:42.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-19T13:05:42.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-19T13:05:42.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485' 2026-02-19T13:05:42.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:42.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:05:42.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672964 2026-02-19T13:05:42.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672964 2026-02-19T13:05:42.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-19T13:05:42.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:42.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-19T13:05:42.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:42.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:05:42.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836485 2026-02-19T13:05:42.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:42.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836485 2026-02-19T13:05:42.739 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836485 2026-02-19T13:05:42.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-19T13:05:42.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:42.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-19T13:05:42.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:42.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672964 2026-02-19T13:05:42.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:42.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:05:42.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672964 2026-02-19T13:05:42.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:42.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672964 2026-02-19T13:05:42.911 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672964 2026-02-19T13:05:42.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-19T13:05:42.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:05:43.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-19T13:05:43.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:05:43.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:43.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:43.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:05:43.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:05:43.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:05:43.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:05:43.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:05:43.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:05:43.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:05:43.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:05:43.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:05:43.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:05:43.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:43.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:43.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:05:43.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:05:43.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:05:43.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6115: TEST_corrupt_snapset_scrub_rep: seq 1 2 2026-02-19T13:05:43.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6115: TEST_corrupt_snapset_scrub_rep: for i in $(seq 1 $total_objs) 2026-02-19T13:05:43.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6116: TEST_corrupt_snapset_scrub_rep: objname=ROBJ1 2026-02-19T13:05:43.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6117: TEST_corrupt_snapset_scrub_rep: add_something td/osd-scrub-repair csr_pool ROBJ1 2026-02-19T13:05:43.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:05:43.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:05:43.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ1 2026-02-19T13:05:43.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:05:43.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:05:43.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:05:43.835 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:05:43.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:05:44.040 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:05:44.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:05:44.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:05:44.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:05:44.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6119: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool setomapheader ROBJ1 hdr-ROBJ1 2026-02-19T13:05:44.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6120: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool setomapval ROBJ1 key-ROBJ1 val-ROBJ1 2026-02-19T13:05:44.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6115: TEST_corrupt_snapset_scrub_rep: for i in $(seq 1 $total_objs) 2026-02-19T13:05:44.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6116: TEST_corrupt_snapset_scrub_rep: objname=ROBJ2 2026-02-19T13:05:44.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6117: TEST_corrupt_snapset_scrub_rep: add_something td/osd-scrub-repair csr_pool ROBJ2 2026-02-19T13:05:44.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:05:44.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:05:44.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ2 2026-02-19T13:05:44.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:05:44.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:05:44.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:05:44.329 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:05:44.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:05:44.534 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:05:44.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:05:44.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:05:44.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ2 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:05:44.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6119: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool setomapheader ROBJ2 hdr-ROBJ2 2026-02-19T13:05:44.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6120: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool setomapval ROBJ2 key-ROBJ2 val-ROBJ2 2026-02-19T13:05:44.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6123: TEST_corrupt_snapset_scrub_rep: get_pg csr_pool ROBJ0 2026-02-19T13:05:44.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=csr_pool 2026-02-19T13:05:44.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=ROBJ0 2026-02-19T13:05:44.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map csr_pool ROBJ0 2026-02-19T13:05:44.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:05:44.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6123: TEST_corrupt_snapset_scrub_rep: local pg=3.0 2026-02-19T13:05:44.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6124: TEST_corrupt_snapset_scrub_rep: get_primary csr_pool ROBJ0 2026-02-19T13:05:44.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=csr_pool 2026-02-19T13:05:44.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=ROBJ0 2026-02-19T13:05:44.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map csr_pool ROBJ0 2026-02-19T13:05:44.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:05:44.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6124: TEST_corrupt_snapset_scrub_rep: local primary=1 2026-02-19T13:05:44.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6126: TEST_corrupt_snapset_scrub_rep: rados -p csr_pool mksnap snap1 2026-02-19T13:05:45.010 INFO:tasks.workunit.client.0.trial144.stdout:created pool csr_pool snap snap1 2026-02-19T13:05:45.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6127: TEST_corrupt_snapset_scrub_rep: echo -n head_of_snapshot_data 2026-02-19T13:05:45.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6129: TEST_corrupt_snapset_scrub_rep: seq 1 2 2026-02-19T13:05:45.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6129: TEST_corrupt_snapset_scrub_rep: for i in $(seq 1 $total_objs) 2026-02-19T13:05:45.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6130: TEST_corrupt_snapset_scrub_rep: objname=ROBJ1 2026-02-19T13:05:45.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6133: TEST_corrupt_snapset_scrub_rep: expr 1 % 2 2026-02-19T13:05:45.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6133: TEST_corrupt_snapset_scrub_rep: local osd=1 2026-02-19T13:05:45.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6135: TEST_corrupt_snapset_scrub_rep: case $i in 2026-02-19T13:05:45.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6137: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool put ROBJ1 td/osd-scrub-repair/change 2026-02-19T13:05:45.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6138: TEST_corrupt_snapset_scrub_rep: objectstore_tool td/osd-scrub-repair 1 --head ROBJ1 clear-snapset corrupt 2026-02-19T13:05:45.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:05:45.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:05:45.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:05:45.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:05:45.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 --head ROBJ1 clear-snapset corrupt 2026-02-19T13:05:45.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:05:45.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:05:45.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:05:45.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:05:45.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:05:45.039 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:05:45.039 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:05:45.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:05:45.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:05:45.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:05:45.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:05:45.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 --head ROBJ1 clear-snapset corrupt 2026-02-19T13:05:45.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:05:45.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:05:45.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:05:45.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:05:45.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:05:45.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 --head ROBJ1 clear-snapset corrupt 2026-02-19T13:05:45.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-19T13:05:45.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:05:45.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:05:45.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:05:45.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:05:45.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:05:45.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:05:45.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:05:45.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:05:45.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:05:45.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:05:45.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:05:45.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:05:45.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:05:45.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:05:45.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:05:45.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:05:45.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:05:45.648 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:05:45.648 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:45.648 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:45.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:05:45.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:05:45.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:05:45.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:05:45.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:05:45.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:05:45.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:05:45.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:05:45.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:05:45.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:05:45.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:05:45.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:05:45.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:05:45.651 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:05:45.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:05:45.651 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:05:45.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:05:45.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:05:45.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:05:45.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:05:45.671 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:45.669+0000 7f6aff79aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:45.672 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:45.669+0000 7f6aff79aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:45.673 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:45.669+0000 7f6aff79aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:45.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:05:45.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:05:45.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:05:45.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:05:45.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:05:45.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:05:45.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:05:45.812 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:05:45.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:05:45.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:05:45.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:05:46.004 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:46.001+0000 7f6aff79aa40 -1 Falling back to public interface 2026-02-19T13:05:46.546 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:46.541+0000 7f6aff79aa40 -1 osd.1 29 log_to_monitors true 2026-02-19T13:05:46.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:05:46.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:05:46.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:05:46.971 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:05:46.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:05:46.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:05:47.132 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 33 up_thru 33 down_at 30 last_clean_interval [10,29) [v2:127.0.0.1:6810/3586448231,v1:127.0.0.1:6811/3586448231] [v2:127.0.0.1:6812/3586448231,v1:127.0.0.1:6813/3586448231] exists,up 0c152b5d-e631-47a6-bf8a-b2062d68495d 2026-02-19T13:05:47.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:05:47.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:05:47.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:05:47.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:05:47.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:05:47.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:05:47.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:05:47.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:05:47.134 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:05:47.134 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:05:47.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:05:47.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:05:47.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:05:47.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:05:47.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:05:47.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:05:47.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:05:47.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:05:47.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:05:47.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:05:47.411 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:05:47.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:05:47.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:47.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:05:47.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-19T13:05:47.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-19T13:05:47.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-19T13:05:47.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:47.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:05:47.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920770 2026-02-19T13:05:47.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920770 2026-02-19T13:05:47.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-141733920770' 2026-02-19T13:05:47.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:47.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-19T13:05:47.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:47.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:05:47.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-19T13:05:47.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:47.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-19T13:05:47.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-19T13:05:47.563 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836487 2026-02-19T13:05:47.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:47.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T13:05:47.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:05:48.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:05:48.734 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:48.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-19T13:05:48.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:48.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-141733920770 2026-02-19T13:05:48.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:48.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:05:48.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-141733920770 2026-02-19T13:05:48.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:48.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920770 2026-02-19T13:05:48.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 141733920770' 2026-02-19T13:05:48.899 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 141733920770 2026-02-19T13:05:48.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:05:49.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920770 -lt 141733920770 2026-02-19T13:05:49.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:05:49.070 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:49.070 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:49.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:05:49.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:05:49.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:05:49.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:05:49.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:05:49.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:05:49.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:05:49.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:05:49.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:05:49.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:05:49.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:49.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:49.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:05:49.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:05:49.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:05:49.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6129: TEST_corrupt_snapset_scrub_rep: for i in $(seq 1 $total_objs) 2026-02-19T13:05:49.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6130: TEST_corrupt_snapset_scrub_rep: objname=ROBJ2 2026-02-19T13:05:49.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6133: TEST_corrupt_snapset_scrub_rep: expr 2 % 2 2026-02-19T13:05:49.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6133: TEST_corrupt_snapset_scrub_rep: local osd=0 2026-02-19T13:05:49.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6135: TEST_corrupt_snapset_scrub_rep: case $i in 2026-02-19T13:05:49.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6142: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool put ROBJ2 td/osd-scrub-repair/change 2026-02-19T13:05:49.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6143: TEST_corrupt_snapset_scrub_rep: objectstore_tool td/osd-scrub-repair 0 --head ROBJ2 clear-snapset corrupt 2026-02-19T13:05:49.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:05:49.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:05:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:05:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:05:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 --head ROBJ2 clear-snapset corrupt 2026-02-19T13:05:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:05:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:05:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:05:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:05:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:05:49.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:05:49.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:05:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:05:49.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:05:49.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:05:49.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:05:49.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 --head ROBJ2 clear-snapset corrupt 2026-02-19T13:05:49.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:05:49.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:05:49.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:05:49.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:05:49.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:05:49.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 --head ROBJ2 clear-snapset corrupt 2026-02-19T13:05:50.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:05:50.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:05:50.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:05:50.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:05:50.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:05:50.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:05:50.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:05:50.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:05:50.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:05:50.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:05:50.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:05:50.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:05:50.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:05:50.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:05:50.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:05:50.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:05:50.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:05:50.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:05:50.226 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:05:50.226 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:05:50.227 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:05:50.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:05:50.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:05:50.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:05:50.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:05:50.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:05:50.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:05:50.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:05:50.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:05:50.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:05:50.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:05:50.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:05:50.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:05:50.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:05:50.229 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:05:50.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:05:50.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:05:50.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:05:50.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:05:50.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:05:50.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:05:50.246 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:50.241+0000 7ff4c88bfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:50.247 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:50.245+0000 7ff4c88bfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:50.248 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:50.245+0000 7ff4c88bfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:05:50.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:05:50.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:05:50.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:05:50.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:05:50.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:05:50.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:05:50.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:05:50.384 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:05:50.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:05:50.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:05:50.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:05:50.612 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:50.609+0000 7ff4c88bfa40 -1 Falling back to public interface 2026-02-19T13:05:51.175 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:05:51.169+0000 7ff4c88bfa40 -1 osd.0 35 log_to_monitors true 2026-02-19T13:05:51.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:05:51.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:05:51.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:05:51.551 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:05:51.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:05:51.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:05:51.710 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [5,35) [v2:127.0.0.1:6802/1047503724,v1:127.0.0.1:6803/1047503724] [v2:127.0.0.1:6804/1047503724,v1:127.0.0.1:6805/1047503724] exists,up d342b0c5-b1ff-4f1a-a794-f47523dc01df 2026-02-19T13:05:51.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:05:51.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:05:51.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:05:51.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:05:51.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:05:51.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:05:51.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:05:51.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:05:51.711 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:05:51.711 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:05:51.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:05:51.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:05:51.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:05:51.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:05:51.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:05:51.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:05:51.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:05:51.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:05:51.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:05:51.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:05:51.980 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:05:51.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:05:51.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:51.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:05:52.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724546 2026-02-19T13:05:52.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724546 2026-02-19T13:05:52.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724546' 2026-02-19T13:05:52.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:52.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:05:52.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920772 2026-02-19T13:05:52.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920772 2026-02-19T13:05:52.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724546 1-141733920772' 2026-02-19T13:05:52.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:52.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-167503724546 2026-02-19T13:05:52.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:52.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:05:52.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-167503724546 2026-02-19T13:05:52.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:52.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724546 2026-02-19T13:05:52.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 167503724546' 2026-02-19T13:05:52.134 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 167503724546 2026-02-19T13:05:52.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:52.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 167503724546 2026-02-19T13:05:52.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:05:53.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:05:53.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:53.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724546 -lt 167503724546 2026-02-19T13:05:53.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:53.466 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-141733920772 2026-02-19T13:05:53.466 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:53.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:05:53.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-141733920772 2026-02-19T13:05:53.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:53.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920772 2026-02-19T13:05:53.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 141733920772' 2026-02-19T13:05:53.470 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 141733920772 2026-02-19T13:05:53.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:05:53.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920772 -lt 141733920772 2026-02-19T13:05:53.641 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:05:53.641 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:53.641 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:53.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:05:53.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:05:53.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:05:53.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:05:53.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:05:53.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:05:53.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:05:53.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:05:53.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:05:53.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:05:53.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:05:53.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:05:54.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:05:54.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:05:54.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:05:54.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6148: TEST_corrupt_snapset_scrub_rep: rm td/osd-scrub-repair/change 2026-02-19T13:05:54.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6150: TEST_corrupt_snapset_scrub_rep: pg_scrub 3.0 2026-02-19T13:05:54.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=3.0 2026-02-19T13:05:54.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 3.0 2026-02-19T13:05:54.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=3.0 2026-02-19T13:05:54.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:05:54.196 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:05:54.197 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:05:54.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:05:54.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:05:54.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:05:54.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:05:54.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:05:54.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:05:54.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:05:54.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:05:54.511 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:05:54.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:05:54.667 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:05:54.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:05:54.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:54.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:05:54.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724547 2026-02-19T13:05:54.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724547 2026-02-19T13:05:54.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724547' 2026-02-19T13:05:54.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:05:54.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:05:54.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920773 2026-02-19T13:05:54.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920773 2026-02-19T13:05:54.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724547 1-141733920773' 2026-02-19T13:05:54.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:54.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-167503724547 2026-02-19T13:05:54.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:54.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:05:54.816 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-167503724547 2026-02-19T13:05:54.817 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:54.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724547 2026-02-19T13:05:54.819 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 167503724547 2026-02-19T13:05:54.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 167503724547' 2026-02-19T13:05:54.819 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:05:54.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-19T13:05:54.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:05:54.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-141733920773 2026-02-19T13:05:54.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:05:54.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:05:54.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-141733920773 2026-02-19T13:05:54.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:05:54.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920773 2026-02-19T13:05:54.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 141733920773' 2026-02-19T13:05:54.980 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 141733920773 2026-02-19T13:05:54.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:05:55.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920772 -lt 141733920773 2026-02-19T13:05:55.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:05:56.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:05:56.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:05:56.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920772 -lt 141733920773 2026-02-19T13:05:56.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:05:57.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:05:57.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:05:57.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920774 -lt 141733920773 2026-02-19T13:05:57.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:05:57.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-19T13:05:57.468 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 3.0 loop 0 2026-02-19T13:05:57.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 3.0 2026-02-19T13:05:57.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=3.0 2026-02-19T13:05:57.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:05:57.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 3.0 query 2026-02-19T13:05:57.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:05:57.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-19T13:05:57.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:05:57.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:05:57.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:05:57.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 3.0 2026-02-19T13:05:57.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:05:57.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:05:57.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:05:57.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:05:57.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-19T13:05:41.290480+0000 2026-02-19T13:05:57.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 3.0 2026-02-19T13:05:57.849 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0 on osd.1 to scrub 2026-02-19T13:05:57.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 3.0 2026-02-19T13:05:41.290480+0000 2026-02-19T13:05:57.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:05:57.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:05:41.290480+0000 2026-02-19T13:05:57.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:05:57.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:05:57.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:05:57.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:05:57.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:05:57.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:05:57.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:05:57.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:05:58.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:05:41.290480+0000 '>' 2026-02-19T13:05:41.290480+0000 2026-02-19T13:05:58.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:05:59.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:05:59.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:05:59.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:05:59.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:05:59.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:05:59.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:05:59.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:05:59.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:05:41.290480+0000 '>' 2026-02-19T13:05:41.290480+0000 2026-02-19T13:05:59.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:06:00.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:06:00.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:06:00.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:06:00.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:06:00.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:06:00.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:06:00.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:06:00.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:05:41.290480+0000 '>' 2026-02-19T13:05:41.290480+0000 2026-02-19T13:06:00.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:06:01.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:06:01.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:06:01.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:06:01.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:06:01.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:06:01.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:06:01.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:06:01.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:05:41.290480+0000 '>' 2026-02-19T13:05:41.290480+0000 2026-02-19T13:06:01.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:06:02.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:06:02.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:06:02.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:06:02.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:06:02.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:06:02.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:06:02.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:06:02.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:05:41.290480+0000 '>' 2026-02-19T13:05:41.290480+0000 2026-02-19T13:06:02.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:06:03.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:06:03.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:06:03.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:06:03.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:06:03.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:06:03.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:06:03.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:06:03.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:05:58.713879+0000 '>' 2026-02-19T13:05:41.290480+0000 2026-02-19T13:06:03.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:06:03.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6152: TEST_corrupt_snapset_scrub_rep: rados list-inconsistent-pg csr_pool 2026-02-19T13:06:03.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6154: TEST_corrupt_snapset_scrub_rep: jq '. | length' td/osd-scrub-repair/json 2026-02-19T13:06:03.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6154: TEST_corrupt_snapset_scrub_rep: test 1 = 1 2026-02-19T13:06:03.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6156: TEST_corrupt_snapset_scrub_rep: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-19T13:06:03.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6156: TEST_corrupt_snapset_scrub_rep: test 3.0 = 3.0 2026-02-19T13:06:03.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6158: TEST_corrupt_snapset_scrub_rep: rados list-inconsistent-obj 3.0 2026-02-19T13:06:03.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6160: TEST_corrupt_snapset_scrub_rep: jq 'def walk(f): 2026-02-19T13:06:03.889 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:06:03.890 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:06:03.890 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:06:03.890 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:06:03.890 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:06:03.890 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:06:03.890 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:06:03.890 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:06:03.890 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:06:03.890 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:06:03.890 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:06:03.890 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-19T13:06:03.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6160: TEST_corrupt_snapset_scrub_rep: jq .inconsistents 2026-02-19T13:06:03.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6160: TEST_corrupt_snapset_scrub_rep: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:06:03.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6329: TEST_corrupt_snapset_scrub_rep: jq 'def walk(f): 2026-02-19T13:06:03.908 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:06:03.908 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:06:03.908 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:06:03.908 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:06:03.908 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:06:03.909 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:06:03.909 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:06:03.909 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:06:03.909 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:06:03.909 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:06:03.909 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:06:03.909 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-19T13:06:03.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6329: TEST_corrupt_snapset_scrub_rep: jq .inconsistents 2026-02-19T13:06:03.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6329: TEST_corrupt_snapset_scrub_rep: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:06:03.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6330: TEST_corrupt_snapset_scrub_rep: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-19T13:06:03.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-19T13:06:03.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6331: TEST_corrupt_snapset_scrub_rep: test no = yes 2026-02-19T13:06:03.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6336: TEST_corrupt_snapset_scrub_rep: test '' = yes 2026-02-19T13:06:03.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6341: TEST_corrupt_snapset_scrub_rep: ERRORS=0 2026-02-19T13:06:03.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6342: TEST_corrupt_snapset_scrub_rep: declare -a err_strings 2026-02-19T13:06:03.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6343: TEST_corrupt_snapset_scrub_rep: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid [0-9]*:.*:::ROBJ1:head : snapset inconsistent' 2026-02-19T13:06:03.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6344: TEST_corrupt_snapset_scrub_rep: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid [0-9]*:.*:::ROBJ2:head : snapset inconsistent' 2026-02-19T13:06:03.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6345: TEST_corrupt_snapset_scrub_rep: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 [0-9]*:.*:::ROBJ1:1 : is an unexpected clone' 2026-02-19T13:06:03.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6346: TEST_corrupt_snapset_scrub_rep: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub : stat mismatch, got 3/4 objects, 1/2 clones, 3/4 dirty, 3/4 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 49/56 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' 2026-02-19T13:06:03.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6347: TEST_corrupt_snapset_scrub_rep: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 0 missing, 2 inconsistent objects' 2026-02-19T13:06:03.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6348: TEST_corrupt_snapset_scrub_rep: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 4 errors' 2026-02-19T13:06:03.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6350: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-19T13:06:03.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6352: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid [0-9]*:.*:::ROBJ1:head : snapset inconsistent' td/osd-scrub-repair/osd.1.log 2026-02-19T13:06:03.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6350: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-19T13:06:03.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6352: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid [0-9]*:.*:::ROBJ2:head : snapset inconsistent' td/osd-scrub-repair/osd.1.log 2026-02-19T13:06:03.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6350: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-19T13:06:03.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6352: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 [0-9]*:.*:::ROBJ1:1 : is an unexpected clone' td/osd-scrub-repair/osd.1.log 2026-02-19T13:06:03.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6350: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-19T13:06:03.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6352: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub : stat mismatch, got 3/4 objects, 1/2 clones, 3/4 dirty, 3/4 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 49/56 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' td/osd-scrub-repair/osd.1.log 2026-02-19T13:06:03.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6350: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-19T13:06:03.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6352: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 0 missing, 2 inconsistent objects' td/osd-scrub-repair/osd.1.log 2026-02-19T13:06:03.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6350: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-19T13:06:03.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6352: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 4 errors' td/osd-scrub-repair/osd.1.log 2026-02-19T13:06:03.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6359: TEST_corrupt_snapset_scrub_rep: '[' 0 '!=' 0 ']' 2026-02-19T13:06:03.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6365: TEST_corrupt_snapset_scrub_rep: ceph osd pool rm csr_pool csr_pool --yes-i-really-really-mean-it 2026-02-19T13:06:04.157 INFO:tasks.workunit.client.0.trial144.stderr:pool 'csr_pool' removed 2026-02-19T13:06:04.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T13:06:04.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:06:04.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:06:04.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:06:04.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:06:04.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:06:04.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:06:04.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:06:04.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:06:04.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:06:04.279 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:06:04.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:06:04.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:06:04.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:06:04.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:06:04.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:06:04.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:06:04.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:06:04.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:06:04.285 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:06:04.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:06:04.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:06:04.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:06:04.298 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:06:04.298 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:04.298 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:04.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:06:04.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:06:04.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:06:04.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T13:06:04.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T13:06:04.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T13:06:04.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T13:06:04.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:06:04.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:06:04.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:06:04.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:06:04.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:06:04.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:06:04.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:06:04.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:06:04.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:06:04.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:06:04.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:06:04.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:06:04.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:06:04.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:06:04.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:06:04.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:06:04.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:06:04.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:06:04.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:06:04.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:06:04.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:06:04.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:06:04.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:06:04.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:04.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:04.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:06:04.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:06:04.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:06:04.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T13:06:04.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:06:04.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:04.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:04.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T13:06:04.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:06:04.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:06:04.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:06:04.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T13:06:04.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_dual_store_replicated_cluster td/osd-scrub-repair 2026-02-19T13:06:04.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6432: TEST_dual_store_replicated_cluster: local dir=td/osd-scrub-repair 2026-02-19T13:06:04.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6433: TEST_dual_store_replicated_cluster: local poolname=csr_pool 2026-02-19T13:06:04.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6434: TEST_dual_store_replicated_cluster: local total_objs=19 2026-02-19T13:06:04.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6435: TEST_dual_store_replicated_cluster: local extr_dbg=1 2026-02-19T13:06:04.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6437: TEST_dual_store_replicated_cluster: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-19T13:06:04.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T13:06:04.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:06:04.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:06:04.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:06:04.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T13:06:04.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-19T13:06:04.342 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:06:04.342 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:06:04.342 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:06:04.342 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:06:04.343 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:04.343 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:04.343 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:04.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-19T13:06:04.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:06:04.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:06:04.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:06:04.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:06:04.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:06:04.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:06:04.374 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:06:04.374 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:06:04.374 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:06:04.375 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:06:04.375 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:04.375 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:04.375 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:06:04.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:06:04.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T13:06:04.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:06:04.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:06:04.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:06:04.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:06:04.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:06:04.433 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:06:04.433 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:06:04.433 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:06:04.434 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:06:04.434 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:04.435 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:04.435 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:06:04.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:06:04.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T13:06:04.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6438: TEST_dual_store_replicated_cluster: run_mgr td/osd-scrub-repair x --mgr_stats_period=1 2026-02-19T13:06:04.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T13:06:04.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:06:04.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:06:04.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:06:04.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T13:06:04.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:06:04.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:06:04.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:06:04.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:06:04.586 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:06:04.586 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:04.586 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:04.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:04.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:06:04.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-19T13:06:04.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6439: TEST_dual_store_replicated_cluster: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 ' 2026-02-19T13:06:04.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6440: TEST_dual_store_replicated_cluster: ceph_osd_args+='--osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 ' 2026-02-19T13:06:04.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6441: TEST_dual_store_replicated_cluster: ceph_osd_args+='--osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 ' 2026-02-19T13:06:04.607 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6442: TEST_dual_store_replicated_cluster: seq 0 1 2026-02-19T13:06:04.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6442: TEST_dual_store_replicated_cluster: for osd in $(seq 0 1) 2026-02-19T13:06:04.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6444: TEST_dual_store_replicated_cluster: run_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:06:04.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:06:04.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:06:04.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:06:04.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:06:04.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:06:04.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:06:04.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:06:04.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:06:04.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:06:04.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:06:04.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:06:04.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:06:04.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:06:04.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:06:04.611 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:06:04.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:06:04.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:06:04.612 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:06:04.612 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:04.612 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:04.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:04.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:04.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:06:04.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:06:04.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:06:04.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:06:04.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:06:04.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:06:04.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:06:04.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:06:04.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:06:04.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:06:04.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:06:04.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:06:04.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:06:04.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fe651040-8a56-4e0e-9bdd-37b3fbeea15d 2026-02-19T13:06:04.614 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 fe651040-8a56-4e0e-9bdd-37b3fbeea15d 2026-02-19T13:06:04.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 fe651040-8a56-4e0e-9bdd-37b3fbeea15d' 2026-02-19T13:06:04.615 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:06:04.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC8CpdpKI47JRAAmg4bpLcctfLTj9wqPKHxmw== 2026-02-19T13:06:04.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC8CpdpKI47JRAAmg4bpLcctfLTj9wqPKHxmw=="}' 2026-02-19T13:06:04.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fe651040-8a56-4e0e-9bdd-37b3fbeea15d -i td/osd-scrub-repair/0/new.json 2026-02-19T13:06:04.711 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:06:04.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T13:06:04.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 --mkfs --key AQC8CpdpKI47JRAAmg4bpLcctfLTj9wqPKHxmw== --osd-uuid fe651040-8a56-4e0e-9bdd-37b3fbeea15d 2026-02-19T13:06:04.739 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:04.737+0000 7f1ed8760a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:04.740 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:04.737+0000 7f1ed8760a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:04.741 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:04.737+0000 7f1ed8760a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:04.741 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:04.737+0000 7f1ed8760a40 -1 bdev(0x562485162c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:06:04.742 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:04.737+0000 7f1ed8760a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T13:06:05.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T13:06:05.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:06:05.538 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:06:05.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:06:05.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:06:05.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:06:05.743 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:06:05.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:06:05.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:06:05.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:06:05.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:06:05.762 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:05.757+0000 7f640f0b1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:05.763 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:05.761+0000 7f640f0b1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:05.764 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:05.761+0000 7f640f0b1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:05.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:06:05.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:06:05.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:06:05.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:06:05.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:06:05.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:06:05.903 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:06:05.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:06:05.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:06:05.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:06:06.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:06:06.144 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:06.141+0000 7f640f0b1a40 -1 Falling back to public interface 2026-02-19T13:06:06.701 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:06.697+0000 7f640f0b1a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:06:07.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:06:07.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:06:07.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:06:07.064 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:06:07.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:06:07.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:06:07.222 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3151395068,v1:127.0.0.1:6803/3151395068] [v2:127.0.0.1:6804/3151395068,v1:127.0.0.1:6805/3151395068] exists,up fe651040-8a56-4e0e-9bdd-37b3fbeea15d 2026-02-19T13:06:07.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:06:07.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:06:07.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:06:07.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6442: TEST_dual_store_replicated_cluster: for osd in $(seq 0 1) 2026-02-19T13:06:07.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6444: TEST_dual_store_replicated_cluster: run_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:06:07.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:06:07.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:06:07.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:06:07.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:06:07.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:06:07.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:06:07.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:06:07.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:06:07.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:06:07.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:06:07.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:06:07.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:06:07.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:06:07.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:06:07.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:06:07.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:06:07.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:06:07.228 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:06:07.228 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:07.228 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:07.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:07.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:07.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:06:07.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:06:07.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:06:07.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:06:07.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:06:07.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:06:07.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:06:07.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:06:07.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:06:07.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:06:07.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:06:07.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:06:07.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:06:07.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:06:07.230 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:06:07.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 c8f44563-f119-46be-a0f5-9bd991c666ec' 2026-02-19T13:06:07.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:06:07.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC/CpdpN35mDhAAs8iUpBtfdkBPsgih2bsCtw== 2026-02-19T13:06:07.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC/CpdpN35mDhAAs8iUpBtfdkBPsgih2bsCtw=="}' 2026-02-19T13:06:07.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c8f44563-f119-46be-a0f5-9bd991c666ec -i td/osd-scrub-repair/1/new.json 2026-02-19T13:06:07.409 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:06:07.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T13:06:07.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 --mkfs --key AQC/CpdpN35mDhAAs8iUpBtfdkBPsgih2bsCtw== --osd-uuid c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:06:07.439 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:07.437+0000 7f0ffe50ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:07.440 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:07.437+0000 7f0ffe50ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:07.441 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:07.437+0000 7f0ffe50ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:07.442 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:07.437+0000 7f0ffe50ba40 -1 bdev(0x5627b7480c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:06:07.442 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:07.437+0000 7f0ffe50ba40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T13:06:08.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T13:06:08.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:06:08.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:06:08.095 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:06:08.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:06:08.294 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:06:08.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:06:08.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:06:08.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:06:08.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:06:08.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:06:08.312 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:08.309+0000 7fe515264a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:08.314 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:08.309+0000 7fe515264a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:08.315 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:08.313+0000 7fe515264a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:08.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:06:08.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:06:08.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:06:08.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:06:08.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:06:08.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:06:08.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:06:08.452 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:06:08.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:06:08.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:06:08.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:06:08.704 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:08.701+0000 7fe515264a40 -1 Falling back to public interface 2026-02-19T13:06:09.285 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:09.281+0000 7fe515264a40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:06:09.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:06:09.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:06:09.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:06:09.613 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:06:09.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:06:09.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:06:09.769 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/753681132,v1:127.0.0.1:6811/753681132] [v2:127.0.0.1:6812/753681132,v1:127.0.0.1:6813/753681132] exists,up c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:06:09.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:06:09.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:06:09.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:06:09.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6447: TEST_dual_store_replicated_cluster: create_rbd_pool 2026-02-19T13:06:09.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T13:06:09.916 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T13:06:09.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T13:06:09.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:06:10.138 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T13:06:10.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:06:11.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T13:06:11.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6448: TEST_dual_store_replicated_cluster: wait_for_clean 2026-02-19T13:06:11.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:06:11.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:06:11.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:06:11.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:06:11.452 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:06:11.452 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:06:11.452 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:06:11.452 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:06:11.452 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:06:11.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:06:11.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:06:11.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:06:11.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:06:11.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:06:11.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:06:11.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:06:11.731 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:06:11.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:06:11.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:11.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:06:11.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-19T13:06:11.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-19T13:06:11.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485' 2026-02-19T13:06:11.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:11.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:06:11.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672963 2026-02-19T13:06:11.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672963 2026-02-19T13:06:11.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-42949672963' 2026-02-19T13:06:11.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:06:11.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-19T13:06:11.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:06:11.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:06:11.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836485 2026-02-19T13:06:11.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:06:11.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836485 2026-02-19T13:06:11.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-19T13:06:11.882 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836485 2026-02-19T13:06:11.882 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:12.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-19T13:06:12.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:06:13.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:06:13.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:13.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836485 2026-02-19T13:06:13.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:06:13.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672963 2026-02-19T13:06:13.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:06:13.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:06:13.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672963 2026-02-19T13:06:13.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:06:13.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672963 2026-02-19T13:06:13.214 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672963 2026-02-19T13:06:13.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672963' 2026-02-19T13:06:13.214 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:06:13.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672964 -lt 42949672963 2026-02-19T13:06:13.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:06:13.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:06:13.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:06:13.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T13:06:13.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:06:13.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:06:13.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:06:13.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:06:13.571 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:06:13.571 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:06:13.571 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:06:13.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T13:06:13.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:06:13.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:06:13.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:06:13.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T13:06:13.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:06:13.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:06:13.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6450: TEST_dual_store_replicated_cluster: create_pool foo 1 2026-02-19T13:06:13.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 1 2026-02-19T13:06:14.124 INFO:tasks.workunit.client.0.trial144.stderr:pool 'foo' created 2026-02-19T13:06:14.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:06:15.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6451: TEST_dual_store_replicated_cluster: create_pool csr_pool 1 1 2026-02-19T13:06:15.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create csr_pool 1 1 2026-02-19T13:06:15.344 INFO:tasks.workunit.client.0.trial144.stderr:pool 'csr_pool' created 2026-02-19T13:06:15.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:06:16.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6452: TEST_dual_store_replicated_cluster: wait_for_clean 2026-02-19T13:06:16.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:06:16.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:06:16.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:06:16.357 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:06:16.357 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:06:16.357 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:06:16.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:06:16.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:06:16.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:06:16.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:06:16.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:06:16.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:06:16.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:06:16.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:06:16.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:06:16.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:06:16.629 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:06:16.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:06:16.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:16.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:06:16.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-19T13:06:16.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-19T13:06:16.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491' 2026-02-19T13:06:16.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:16.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:06:16.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672969 2026-02-19T13:06:16.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672969 2026-02-19T13:06:16.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672969' 2026-02-19T13:06:16.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:06:16.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-19T13:06:16.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:06:16.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:06:16.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836491 2026-02-19T13:06:16.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:06:16.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836491 2026-02-19T13:06:16.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-19T13:06:16.782 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836491 2026-02-19T13:06:16.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:16.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836489 -lt 21474836491 2026-02-19T13:06:16.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:06:17.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:06:17.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:18.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836492 -lt 21474836491 2026-02-19T13:06:18.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:06:18.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672969 2026-02-19T13:06:18.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:06:18.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:06:18.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672969 2026-02-19T13:06:18.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:06:18.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672969 2026-02-19T13:06:18.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672969' 2026-02-19T13:06:18.107 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672969 2026-02-19T13:06:18.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:06:18.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672969 -lt 42949672969 2026-02-19T13:06:18.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:06:18.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:06:18.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:06:18.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:06:18.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:06:18.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:06:18.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:06:18.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:06:18.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:06:18.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:06:18.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:06:18.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:06:18.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:06:18.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:06:18.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:06:18.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:06:18.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:06:18.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:06:18.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6454: TEST_dual_store_replicated_cluster: ceph osd pool set csr_pool noscrub 1 2026-02-19T13:06:19.006 INFO:tasks.workunit.client.0.trial144.stderr:set pool 3 noscrub to 1 2026-02-19T13:06:19.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6455: TEST_dual_store_replicated_cluster: ceph osd pool set csr_pool nodeep-scrub 1 2026-02-19T13:06:19.213 INFO:tasks.workunit.client.0.trial144.stderr:set pool 3 nodeep-scrub to 1 2026-02-19T13:06:19.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: seq 1 19 2026-02-19T13:06:19.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:19.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ1 2026-02-19T13:06:19.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ1 2026-02-19T13:06:19.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:19.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:19.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ1 2026-02-19T13:06:19.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:19.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:19.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:19.415 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:19.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:19.616 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:19.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:19.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:19.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:19.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ1 hdr-ROBJ1 2026-02-19T13:06:19.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ1 key-ROBJ1 val-ROBJ1 2026-02-19T13:06:19.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:19.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ2 2026-02-19T13:06:19.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ2 2026-02-19T13:06:19.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:19.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:19.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ2 2026-02-19T13:06:19.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:19.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:19.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:19.911 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:19.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:20.115 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:20.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:20.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:20.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ2 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:20.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ2 hdr-ROBJ2 2026-02-19T13:06:20.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ2 key-ROBJ2 val-ROBJ2 2026-02-19T13:06:20.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:20.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ3 2026-02-19T13:06:20.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ3 2026-02-19T13:06:20.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:20.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:20.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ3 2026-02-19T13:06:20.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:20.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:20.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:20.403 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:20.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:20.606 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:20.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:20.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:20.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ3 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:20.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ3 hdr-ROBJ3 2026-02-19T13:06:20.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ3 key-ROBJ3 val-ROBJ3 2026-02-19T13:06:20.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:20.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ4 2026-02-19T13:06:20.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ4 2026-02-19T13:06:20.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:20.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:20.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ4 2026-02-19T13:06:20.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:20.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:20.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:20.894 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:20.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:21.098 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:21.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:21.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:21.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ4 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:21.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ4 hdr-ROBJ4 2026-02-19T13:06:21.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ4 key-ROBJ4 val-ROBJ4 2026-02-19T13:06:21.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:21.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ5 2026-02-19T13:06:21.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ5 2026-02-19T13:06:21.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:21.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:21.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ5 2026-02-19T13:06:21.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:21.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:21.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:21.385 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:21.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:21.589 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:21.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:21.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:21.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ5 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:21.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ5 hdr-ROBJ5 2026-02-19T13:06:21.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ5 key-ROBJ5 val-ROBJ5 2026-02-19T13:06:21.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:21.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ6 2026-02-19T13:06:21.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ6 2026-02-19T13:06:21.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:21.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:21.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ6 2026-02-19T13:06:21.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:21.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:21.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:21.886 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:21.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:22.090 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:22.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:22.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:22.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ6 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:22.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ6 hdr-ROBJ6 2026-02-19T13:06:22.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ6 key-ROBJ6 val-ROBJ6 2026-02-19T13:06:22.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:22.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ7 2026-02-19T13:06:22.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ7 2026-02-19T13:06:22.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:22.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:22.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ7 2026-02-19T13:06:22.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:22.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:22.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:22.378 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:22.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:22.582 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:22.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:22.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:22.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ7 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:22.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ7 hdr-ROBJ7 2026-02-19T13:06:22.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ7 key-ROBJ7 val-ROBJ7 2026-02-19T13:06:22.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:22.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ8 2026-02-19T13:06:22.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ8 2026-02-19T13:06:22.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:22.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:22.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ8 2026-02-19T13:06:22.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:22.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:22.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:22.872 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:22.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:23.075 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:23.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:23.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:23.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ8 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:23.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ8 hdr-ROBJ8 2026-02-19T13:06:23.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ8 key-ROBJ8 val-ROBJ8 2026-02-19T13:06:23.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:23.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ9 2026-02-19T13:06:23.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ9 2026-02-19T13:06:23.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:23.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:23.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ9 2026-02-19T13:06:23.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:23.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:23.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:23.362 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:23.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:23.566 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:23.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:23.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:23.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ9 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:23.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ9 hdr-ROBJ9 2026-02-19T13:06:23.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ9 key-ROBJ9 val-ROBJ9 2026-02-19T13:06:23.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:23.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ10 2026-02-19T13:06:23.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ10 2026-02-19T13:06:23.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:23.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:23.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ10 2026-02-19T13:06:23.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:23.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:23.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:23.862 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:23.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:24.066 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:24.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:24.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:24.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ10 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:24.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ10 hdr-ROBJ10 2026-02-19T13:06:24.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ10 key-ROBJ10 val-ROBJ10 2026-02-19T13:06:24.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:24.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ11 2026-02-19T13:06:24.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ11 2026-02-19T13:06:24.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:24.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:24.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ11 2026-02-19T13:06:24.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:24.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:24.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:24.350 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:24.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:24.554 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:24.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:24.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:24.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ11 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:24.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ11 hdr-ROBJ11 2026-02-19T13:06:24.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ11 key-ROBJ11 val-ROBJ11 2026-02-19T13:06:24.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:24.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ12 2026-02-19T13:06:24.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ12 2026-02-19T13:06:24.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:24.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:24.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ12 2026-02-19T13:06:24.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:24.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:24.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:24.841 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:24.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:25.046 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:25.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:25.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:25.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ12 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:25.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ12 hdr-ROBJ12 2026-02-19T13:06:25.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ12 key-ROBJ12 val-ROBJ12 2026-02-19T13:06:25.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:25.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ13 2026-02-19T13:06:25.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ13 2026-02-19T13:06:25.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:25.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:25.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ13 2026-02-19T13:06:25.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:25.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:25.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:25.337 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:25.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:25.541 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:25.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:25.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:25.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ13 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:25.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ13 hdr-ROBJ13 2026-02-19T13:06:25.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ13 key-ROBJ13 val-ROBJ13 2026-02-19T13:06:25.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:25.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ14 2026-02-19T13:06:25.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ14 2026-02-19T13:06:25.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:25.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:25.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ14 2026-02-19T13:06:25.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:25.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:25.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:25.832 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:25.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:26.036 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:26.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:26.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:26.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ14 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:26.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ14 hdr-ROBJ14 2026-02-19T13:06:26.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ14 key-ROBJ14 val-ROBJ14 2026-02-19T13:06:26.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:26.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ15 2026-02-19T13:06:26.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ15 2026-02-19T13:06:26.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:26.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:26.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ15 2026-02-19T13:06:26.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:26.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:26.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:26.332 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:26.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:26.537 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:26.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:26.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:26.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ15 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:26.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ15 hdr-ROBJ15 2026-02-19T13:06:26.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ15 key-ROBJ15 val-ROBJ15 2026-02-19T13:06:26.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:26.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ16 2026-02-19T13:06:26.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ16 2026-02-19T13:06:26.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:26.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:26.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ16 2026-02-19T13:06:26.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:26.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:26.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:26.824 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:26.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:27.029 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:27.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:27.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:27.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ16 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:27.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ16 hdr-ROBJ16 2026-02-19T13:06:27.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ16 key-ROBJ16 val-ROBJ16 2026-02-19T13:06:27.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:27.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ17 2026-02-19T13:06:27.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ17 2026-02-19T13:06:27.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:27.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:27.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ17 2026-02-19T13:06:27.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:27.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:27.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:27.311 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:27.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:27.516 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:27.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:27.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:27.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ17 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:27.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ17 hdr-ROBJ17 2026-02-19T13:06:27.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ17 key-ROBJ17 val-ROBJ17 2026-02-19T13:06:27.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:27.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ18 2026-02-19T13:06:27.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ18 2026-02-19T13:06:27.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:27.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:27.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ18 2026-02-19T13:06:27.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:27.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:27.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:27.799 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:27.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:28.004 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:28.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:28.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:28.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ18 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:28.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ18 hdr-ROBJ18 2026-02-19T13:06:28.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ18 key-ROBJ18 val-ROBJ18 2026-02-19T13:06:28.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6457: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:28.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6458: TEST_dual_store_replicated_cluster: objname=ROBJ19 2026-02-19T13:06:28.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6459: TEST_dual_store_replicated_cluster: add_something td/osd-scrub-repair csr_pool ROBJ19 2026-02-19T13:06:28.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:06:28.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-19T13:06:28.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ19 2026-02-19T13:06:28.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:06:28.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:06:28.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:06:28.303 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:06:28.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:06:28.508 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:06:28.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:06:28.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:06:28.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ19 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:06:28.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6461: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapheader ROBJ19 hdr-ROBJ19 2026-02-19T13:06:28.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6462: TEST_dual_store_replicated_cluster: rados --pool csr_pool setomapval ROBJ19 key-ROBJ19 val-ROBJ19 2026-02-19T13:06:28.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6466: TEST_dual_store_replicated_cluster: dd if=/dev/zero of=td/osd-scrub-repair/new.ROBJ19 bs=1024 count=1025 2026-02-19T13:06:28.604 INFO:tasks.workunit.client.0.trial144.stderr:1025+0 records in 2026-02-19T13:06:28.604 INFO:tasks.workunit.client.0.trial144.stderr:1025+0 records out 2026-02-19T13:06:28.604 INFO:tasks.workunit.client.0.trial144.stderr:1049600 bytes (1.0 MB, 1.0 MiB) copied, 0.00414243 s, 253 MB/s 2026-02-19T13:06:28.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6467: TEST_dual_store_replicated_cluster: rados --pool csr_pool put ROBJ19 td/osd-scrub-repair/new.ROBJ19 2026-02-19T13:06:28.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6468: TEST_dual_store_replicated_cluster: rm -f td/osd-scrub-repair/new.ROBJ19 2026-02-19T13:06:28.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6470: TEST_dual_store_replicated_cluster: get_pg csr_pool ROBJ0 2026-02-19T13:06:28.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=csr_pool 2026-02-19T13:06:28.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=ROBJ0 2026-02-19T13:06:28.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map csr_pool ROBJ0 2026-02-19T13:06:28.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:06:28.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6470: TEST_dual_store_replicated_cluster: local pg=3.0 2026-02-19T13:06:28.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6471: TEST_dual_store_replicated_cluster: get_primary csr_pool ROBJ0 2026-02-19T13:06:28.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=csr_pool 2026-02-19T13:06:28.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=ROBJ0 2026-02-19T13:06:28.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map csr_pool ROBJ0 2026-02-19T13:06:28.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:06:28.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6471: TEST_dual_store_replicated_cluster: local primary=1 2026-02-19T13:06:28.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6474: TEST_dual_store_replicated_cluster: get_asok_path osd.0 2026-02-19T13:06:28.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:06:28.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:06:28.959 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:06:28.960 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:28.960 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:28.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.0.asok 2026-02-19T13:06:28.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6474: TEST_dual_store_replicated_cluster: CEPH_ARGS= 2026-02-19T13:06:28.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6474: TEST_dual_store_replicated_cluster: ceph daemon /tmp/ceph-asok.35818/ceph-osd.0.asok config set osd_deep_scrub_update_digest_min_age 0 2026-02-19T13:06:29.014 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:06:29.014 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_deep_scrub_update_digest_min_age = '' (not observed, change may require restart) " 2026-02-19T13:06:29.014 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:06:29.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6476: TEST_dual_store_replicated_cluster: get_asok_path osd.1 2026-02-19T13:06:29.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:06:29.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:06:29.021 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:06:29.022 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:29.022 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:29.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T13:06:29.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6476: TEST_dual_store_replicated_cluster: CEPH_ARGS= 2026-02-19T13:06:29.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6476: TEST_dual_store_replicated_cluster: ceph daemon /tmp/ceph-asok.35818/ceph-osd.1.asok config set osd_deep_scrub_update_digest_min_age 0 2026-02-19T13:06:29.074 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:06:29.074 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_deep_scrub_update_digest_min_age = '' (not observed, change may require restart) " 2026-02-19T13:06:29.074 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:06:29.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6478: TEST_dual_store_replicated_cluster: pg_deep_scrub 3.0 2026-02-19T13:06:29.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_deep_scrub: local pgid=3.0 2026-02-19T13:06:29.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_deep_scrub: wait_for_pg_clean 3.0 2026-02-19T13:06:29.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=3.0 2026-02-19T13:06:29.080 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:06:29.080 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:06:29.080 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:06:29.080 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:06:29.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:06:29.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:06:29.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:06:29.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:06:29.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:06:29.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:06:29.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:06:29.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:06:29.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:06:29.532 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:06:29.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:06:29.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:29.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:06:29.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836505 2026-02-19T13:06:29.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836505 2026-02-19T13:06:29.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505' 2026-02-19T13:06:29.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:29.609 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:06:29.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672983 2026-02-19T13:06:29.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672983 2026-02-19T13:06:29.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836505 1-42949672983' 2026-02-19T13:06:29.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:06:29.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836505 2026-02-19T13:06:29.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:06:29.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:06:29.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836505 2026-02-19T13:06:29.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:06:29.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836505 2026-02-19T13:06:29.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836505' 2026-02-19T13:06:29.685 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836505 2026-02-19T13:06:29.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:29.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836504 -lt 21474836505 2026-02-19T13:06:29.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:06:30.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:06:30.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:31.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836504 -lt 21474836505 2026-02-19T13:06:31.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:06:32.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:06:32.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:32.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836507 -lt 21474836505 2026-02-19T13:06:32.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:06:32.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672983 2026-02-19T13:06:32.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:06:32.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:06:32.182 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672983 2026-02-19T13:06:32.183 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:06:32.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672983 2026-02-19T13:06:32.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672983' 2026-02-19T13:06:32.185 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672983 2026-02-19T13:06:32.185 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:06:32.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672984 -lt 42949672983 2026-02-19T13:06:32.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:06:32.352 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 3.0 loop 0 2026-02-19T13:06:32.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-19T13:06:32.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 3.0 2026-02-19T13:06:32.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=3.0 2026-02-19T13:06:32.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:06:32.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 3.0 query 2026-02-19T13:06:32.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:06:32.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-19T13:06:32.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:06:32.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:06:32.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:06:32.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:06:32.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:06:32.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:06:32.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:06:32.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:06:32.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: local last_scrub=2026-02-19T13:06:15.344560+0000 2026-02-19T13:06:32.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_deep_scrub: ceph pg deep-scrub 3.0 2026-02-19T13:06:32.736 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2026-02-19T13:06:32.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_deep_scrub: wait_for_scrub 3.0 2026-02-19T13:06:15.344560+0000 last_deep_scrub_stamp 2026-02-19T13:06:32.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:06:32.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:06:15.344560+0000 2026-02-19T13:06:32.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-19T13:06:32.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:06:32.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:06:32.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:06:32.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:06:32.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:06:32.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:06:32.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:06:32.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:06:15.344560+0000 '>' 2026-02-19T13:06:15.344560+0000 2026-02-19T13:06:32.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:06:33.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:06:33.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:06:33.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:06:33.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:06:33.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:06:33.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:06:33.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:06:34.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:06:15.344560+0000 '>' 2026-02-19T13:06:15.344560+0000 2026-02-19T13:06:34.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:06:35.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:06:35.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:06:35.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:06:35.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:06:35.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:06:35.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:06:35.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:06:35.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:06:33.452816+0000 '>' 2026-02-19T13:06:15.344560+0000 2026-02-19T13:06:35.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:06:35.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: seq 1 19 2026-02-19T13:06:35.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:35.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ1 2026-02-19T13:06:35.236 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 1 % 2 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=1 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6489: TEST_dual_store_replicated_cluster: local payload=UVWXYZZZ 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6490: TEST_dual_store_replicated_cluster: echo UVWXYZZZ 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6491: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:06:35.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:06:35.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:06:35.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:06:35.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:06:35.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:06:35.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:06:35.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:06:35.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:06:35.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:06:35.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:06:35.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:06:35.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:06:35.725 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#-5:00000000:::deep_scrub_3.0:head#, (61) No data available 2026-02-19T13:06:35.725 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#-5:00000000:::scrub_3.0:head#, (61) No data available 2026-02-19T13:06:35.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:06:35.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:06:35.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:06:35.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:06:35.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:06:35.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:06:35.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:06:35.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:06:35.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:06:35.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:06:35.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:06:35.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:06:35.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:06:35.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:06:35.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:06:35.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:06:35.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:06:35.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:06:35.836 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:06:35.836 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:35.836 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:35.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:35.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:35.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:06:35.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:06:35.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:06:35.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:06:35.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:06:35.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:06:35.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:06:35.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:06:35.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:06:35.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:06:35.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:06:35.838 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:06:35.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:06:35.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:06:35.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:06:35.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:06:35.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:06:35.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:06:35.860 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:35.856+0000 7ffb0ff65a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:35.861 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:35.856+0000 7ffb0ff65a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:35.863 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:35.860+0000 7ffb0ff65a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:36.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:06:36.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:06:36.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:06:36.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:06:36.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:06:36.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:06:36.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:06:36.002 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:06:36.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:06:36.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:06:36.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:06:36.244 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:36.240+0000 7ffb0ff65a40 -1 Falling back to public interface 2026-02-19T13:06:36.951 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:36.944+0000 7ffb0ff65a40 -1 osd.1 63 log_to_monitors true 2026-02-19T13:06:37.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:06:37.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:06:37.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:06:37.167 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:06:37.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:06:37.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:06:37.323 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 67 up_thru 67 down_at 64 last_clean_interval [10,63) [v2:127.0.0.1:6810/1232806264,v1:127.0.0.1:6811/1232806264] [v2:127.0.0.1:6812/1232806264,v1:127.0.0.1:6813/1232806264] exists,up c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:06:37.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:06:37.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:06:37.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:06:37.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:06:37.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:06:37.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:06:37.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:06:37.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:06:37.324 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:06:37.324 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:06:37.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:06:37.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:06:37.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:06:37.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:06:37.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:06:37.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:06:37.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:06:37.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:06:37.440 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:06:37.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:06:37.596 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:06:37.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:06:37.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:37.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:06:37.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836514 2026-02-19T13:06:37.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836514 2026-02-19T13:06:37.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514' 2026-02-19T13:06:37.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:37.671 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:06:37.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808834 2026-02-19T13:06:37.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808834 2026-02-19T13:06:37.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-287762808834' 2026-02-19T13:06:37.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:06:37.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836514 2026-02-19T13:06:37.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:06:37.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:06:37.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836514 2026-02-19T13:06:37.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:06:37.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836514 2026-02-19T13:06:37.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836514' 2026-02-19T13:06:37.747 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836514 2026-02-19T13:06:37.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:37.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836513 -lt 21474836514 2026-02-19T13:06:37.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:06:38.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:06:38.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:39.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836513 -lt 21474836514 2026-02-19T13:06:39.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:06:40.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:06:40.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:40.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836516 -lt 21474836514 2026-02-19T13:06:40.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:06:40.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-287762808834 2026-02-19T13:06:40.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:06:40.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:06:40.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-287762808834 2026-02-19T13:06:40.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:06:40.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=287762808834 2026-02-19T13:06:40.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 287762808834' 2026-02-19T13:06:40.234 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 287762808834 2026-02-19T13:06:40.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:06:40.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 287762808836 -lt 287762808834 2026-02-19T13:06:40.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:06:40.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:06:40.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:06:40.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:06:40.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:06:40.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:06:40.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:06:40.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:06:40.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:06:40.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:06:40.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:06:40.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:06:40.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:06:40.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:06:40.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:06:40.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:06:40.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:06:40.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:06:40.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:40.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ2 2026-02-19T13:06:40.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 2 % 2 2026-02-19T13:06:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=0 2026-02-19T13:06:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:06:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6496: TEST_dual_store_replicated_cluster: local payload=UVWXYZ 2026-02-19T13:06:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6497: TEST_dual_store_replicated_cluster: echo UVWXYZ 2026-02-19T13:06:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6498: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:06:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:06:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:06:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:06:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:06:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:06:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:06:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:06:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:06:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:06:40.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:06:40.937 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:06:40.937 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:06:40.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:06:40.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:06:40.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:06:41.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:06:41.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:06:41.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:06:41.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:06:41.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:06:41.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:06:41.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:06:41.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-19T13:06:41.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:06:41.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:06:41.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:06:41.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:06:41.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:06:41.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:06:41.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:06:41.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:06:41.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:06:41.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:06:41.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:06:41.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:06:41.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:06:41.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:06:41.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:06:41.554 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:06:41.554 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:06:41.554 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:06:41.554 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:06:41.555 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:41.555 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:41.555 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:41.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:41.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:06:41.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:06:41.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:06:41.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:06:41.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:06:41.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:06:41.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:06:41.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:06:41.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:06:41.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:06:41.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:06:41.558 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:06:41.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:06:41.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:06:41.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:06:41.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:06:41.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:06:41.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:06:41.574 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:41.568+0000 7fd114b12a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:41.576 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:41.572+0000 7fd114b12a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:41.577 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:41.572+0000 7fd114b12a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:41.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:06:41.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:06:41.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:06:41.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:06:41.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:06:41.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:06:41.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:06:41.717 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:06:41.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:06:41.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:06:41.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:06:41.956 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:41.952+0000 7fd114b12a40 -1 Falling back to public interface 2026-02-19T13:06:42.577 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:42.572+0000 7fd114b12a40 -1 osd.0 68 log_to_monitors true 2026-02-19T13:06:42.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:06:42.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:06:42.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:06:42.875 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:06:42.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:06:42.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:06:43.032 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 72 up_thru 72 down_at 69 last_clean_interval [5,68) [v2:127.0.0.1:6802/1309440366,v1:127.0.0.1:6803/1309440366] [v2:127.0.0.1:6804/1309440366,v1:127.0.0.1:6805/1309440366] exists,up fe651040-8a56-4e0e-9bdd-37b3fbeea15d 2026-02-19T13:06:43.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:06:43.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:06:43.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:06:43.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:06:43.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:06:43.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:06:43.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:06:43.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:06:43.034 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:06:43.034 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:06:43.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:06:43.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:06:43.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:06:43.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:06:43.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:06:43.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:06:43.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:06:43.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:06:43.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:06:43.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:06:43.307 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:06:43.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:06:43.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:43.308 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:06:43.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645314 2026-02-19T13:06:43.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645314 2026-02-19T13:06:43.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-309237645314' 2026-02-19T13:06:43.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:43.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:06:43.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808841 2026-02-19T13:06:43.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808841 2026-02-19T13:06:43.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-309237645314 1-287762808841' 2026-02-19T13:06:43.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:06:43.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-309237645314 2026-02-19T13:06:43.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:06:43.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:06:43.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-309237645314 2026-02-19T13:06:43.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:06:43.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=309237645314 2026-02-19T13:06:43.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 309237645314' 2026-02-19T13:06:43.460 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 309237645314 2026-02-19T13:06:43.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:43.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 309237645314 2026-02-19T13:06:43.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:06:44.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:06:44.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:44.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 309237645314 2026-02-19T13:06:44.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:06:45.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:06:45.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:45.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645316 -lt 309237645314 2026-02-19T13:06:45.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:06:45.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-287762808841 2026-02-19T13:06:45.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:06:45.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:06:45.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-287762808841 2026-02-19T13:06:45.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:06:45.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=287762808841 2026-02-19T13:06:45.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 287762808841' 2026-02-19T13:06:45.950 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 287762808841 2026-02-19T13:06:45.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:06:46.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 287762808843 -lt 287762808841 2026-02-19T13:06:46.122 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:06:46.123 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:06:46.123 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:06:46.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:06:46.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:06:46.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:06:46.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:06:46.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:06:46.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:06:46.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:06:46.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:06:46.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:06:46.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:06:46.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:06:46.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:06:46.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:06:46.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:06:46.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:06:46.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:46.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ3 2026-02-19T13:06:46.671 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 3 % 2 2026-02-19T13:06:46.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=1 2026-02-19T13:06:46.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:06:46.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6503: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 1 ROBJ3 remove 2026-02-19T13:06:46.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:06:46.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:06:46.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:06:46.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:06:46.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ3 remove 2026-02-19T13:06:46.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:06:46.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:06:46.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:06:46.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:06:46.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:06:46.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:06:46.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:06:46.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:06:46.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:06:46.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:06:46.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:06:46.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ3 remove 2026-02-19T13:06:46.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:06:46.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:06:46.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:06:46.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:06:46.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:06:46.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ3 remove 2026-02-19T13:06:47.195 INFO:tasks.workunit.client.0.trial144.stdout:remove #3:f2a5b2a4:::ROBJ3:head# 2026-02-19T13:06:47.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:06:47.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:06:47.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:06:47.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:06:47.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:06:47.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:06:47.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:06:47.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:06:47.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:06:47.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:06:47.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:06:47.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:06:47.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:06:47.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:06:47.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:06:47.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:06:47.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:06:47.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:06:47.303 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:06:47.303 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:47.303 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:47.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:47.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:47.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:06:47.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:06:47.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:06:47.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:06:47.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:06:47.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:06:47.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:06:47.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:06:47.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:06:47.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:06:47.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:06:47.304 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:06:47.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:06:47.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:06:47.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:06:47.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:06:47.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:06:47.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:06:47.324 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:47.324+0000 7fbb909cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:47.326 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:47.324+0000 7fbb909cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:47.327 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:47.328+0000 7fbb909cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:47.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:06:47.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:06:47.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:06:47.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:06:47.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:06:47.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:06:47.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:06:47.466 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:06:47.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:06:47.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:06:47.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:06:47.668 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:47.668+0000 7fbb909cca40 -1 Falling back to public interface 2026-02-19T13:06:48.319 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:48.320+0000 7fbb909cca40 -1 osd.1 73 log_to_monitors true 2026-02-19T13:06:48.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:06:48.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:06:48.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:06:48.626 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:06:48.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:06:48.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:06:48.780 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 77 up_thru 77 down_at 74 last_clean_interval [67,73) [v2:127.0.0.1:6810/518383096,v1:127.0.0.1:6811/518383096] [v2:127.0.0.1:6812/518383096,v1:127.0.0.1:6813/518383096] exists,up c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:06:48.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:06:48.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:06:48.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:06:48.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:06:48.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:06:48.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:06:48.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:06:48.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:06:48.782 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:06:48.782 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:06:48.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:06:48.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:06:48.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:06:48.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:06:48.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:06:48.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:06:48.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:06:48.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:06:48.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:06:49.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:06:49.062 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:06:49.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:06:49.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:49.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:06:49.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645321 2026-02-19T13:06:49.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645321 2026-02-19T13:06:49.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-309237645321' 2026-02-19T13:06:49.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:49.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:06:49.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=330712481794 2026-02-19T13:06:49.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 330712481794 2026-02-19T13:06:49.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-309237645321 1-330712481794' 2026-02-19T13:06:49.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:06:49.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-309237645321 2026-02-19T13:06:49.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:06:49.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:06:49.210 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-309237645321 2026-02-19T13:06:49.210 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:06:49.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=309237645321 2026-02-19T13:06:49.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 309237645321' 2026-02-19T13:06:49.212 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 309237645321 2026-02-19T13:06:49.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:49.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645320 -lt 309237645321 2026-02-19T13:06:49.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:06:50.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:06:50.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:50.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645320 -lt 309237645321 2026-02-19T13:06:50.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:06:51.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:06:51.537 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:51.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645323 -lt 309237645321 2026-02-19T13:06:51.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:06:51.704 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-330712481794 2026-02-19T13:06:51.704 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:06:51.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:06:51.705 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-330712481794 2026-02-19T13:06:51.705 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:06:51.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=330712481794 2026-02-19T13:06:51.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 330712481794' 2026-02-19T13:06:51.705 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 330712481794 2026-02-19T13:06:51.705 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:06:51.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 330712481796 -lt 330712481794 2026-02-19T13:06:51.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:06:51.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:06:51.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:06:52.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:06:52.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:06:52.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:06:52.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:06:52.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:06:52.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:06:52.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:06:52.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:06:52.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:06:52.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:06:52.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:06:52.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:06:52.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:06:52.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:06:52.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:06:52.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:52.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ4 2026-02-19T13:06:52.416 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 4 % 2 2026-02-19T13:06:52.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=0 2026-02-19T13:06:52.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:06:52.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6508: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-19T13:06:52.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:06:52.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:06:52.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:06:52.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:06:52.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-19T13:06:52.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:06:52.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:06:52.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:06:52.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:06:52.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:06:52.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:06:52.420 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:06:52.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:06:52.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:06:52.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:06:52.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:06:52.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-19T13:06:52.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:06:52.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:06:52.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:06:52.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:06:52.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:06:52.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-19T13:06:53.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:06:53.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:06:53.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:06:53.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:06:53.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:06:53.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:06:53.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:06:53.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:06:53.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:06:53.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:06:53.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:06:53.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:06:53.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:06:53.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:06:53.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:06:53.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:06:53.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:06:53.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:06:53.025 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:06:53.025 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:53.025 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:53.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:53.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:53.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:06:53.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:06:53.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:06:53.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:06:53.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:06:53.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:06:53.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:06:53.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:06:53.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:06:53.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:06:53.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:06:53.028 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:06:53.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:06:53.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:06:53.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:06:53.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:06:53.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:06:53.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:06:53.042 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:53.040+0000 7f64e6db8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:53.044 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:53.044+0000 7f64e6db8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:53.044 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:53.044+0000 7f64e6db8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:53.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:06:53.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:06:53.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:06:53.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:06:53.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:06:53.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:06:53.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:06:53.186 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:06:53.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:06:53.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:06:53.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:06:53.476 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:53.476+0000 7f64e6db8a40 -1 Falling back to public interface 2026-02-19T13:06:54.142 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:54.140+0000 7f64e6db8a40 -1 osd.0 78 log_to_monitors true 2026-02-19T13:06:54.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:06:54.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:06:54.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:06:54.343 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:06:54.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:06:54.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:06:54.500 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 82 up_thru 82 down_at 79 last_clean_interval [72,78) [v2:127.0.0.1:6802/2126866698,v1:127.0.0.1:6803/2126866698] [v2:127.0.0.1:6804/2126866698,v1:127.0.0.1:6805/2126866698] exists,up fe651040-8a56-4e0e-9bdd-37b3fbeea15d 2026-02-19T13:06:54.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:06:54.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:06:54.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:06:54.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:06:54.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:06:54.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:06:54.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:06:54.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:06:54.501 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:06:54.502 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:06:54.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:06:54.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:06:54.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:06:54.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:06:54.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:06:54.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:06:54.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:06:54.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:06:54.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:06:54.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:06:54.787 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:06:54.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:06:54.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:54.787 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:06:54.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=352187318274 2026-02-19T13:06:54.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 352187318274 2026-02-19T13:06:54.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-352187318274' 2026-02-19T13:06:54.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:54.862 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:06:54.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=330712481801 2026-02-19T13:06:54.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 330712481801 2026-02-19T13:06:54.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-352187318274 1-330712481801' 2026-02-19T13:06:54.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:06:54.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-352187318274 2026-02-19T13:06:54.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:06:54.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:06:54.937 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-352187318274 2026-02-19T13:06:54.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:06:54.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=352187318274 2026-02-19T13:06:54.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 352187318274' 2026-02-19T13:06:54.939 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 352187318274 2026-02-19T13:06:54.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:55.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645324 -lt 352187318274 2026-02-19T13:06:55.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:06:56.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:06:56.100 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:56.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 352187318274 -lt 352187318274 2026-02-19T13:06:56.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:06:56.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-330712481801 2026-02-19T13:06:56.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:06:56.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:06:56.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-330712481801 2026-02-19T13:06:56.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:06:56.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=330712481801 2026-02-19T13:06:56.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 330712481801' 2026-02-19T13:06:56.268 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 330712481801 2026-02-19T13:06:56.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:06:56.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 330712481801 -lt 330712481801 2026-02-19T13:06:56.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:06:56.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:06:56.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:06:56.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:06:56.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:06:56.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:06:56.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:06:56.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:06:56.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:06:56.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:06:56.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:06:56.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:06:56.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:06:56.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:06:56.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:06:56.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:06:56.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:06:56.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:06:56.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:06:56.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ5 2026-02-19T13:06:56.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 5 % 2 2026-02-19T13:06:56.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=1 2026-02-19T13:06:56.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:06:56.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6513: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 1 ROBJ5 rm-omap key-ROBJ5 2026-02-19T13:06:56.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:06:56.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:06:56.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:06:56.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:06:56.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ5 rm-omap key-ROBJ5 2026-02-19T13:06:56.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:06:56.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:06:56.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:06:56.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:06:56.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:06:56.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:06:56.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:06:56.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:06:56.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:06:56.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:06:57.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:06:57.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ5 rm-omap key-ROBJ5 2026-02-19T13:06:57.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:06:57.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:06:57.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:06:57.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:06:57.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:06:57.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ5 rm-omap key-ROBJ5 2026-02-19T13:06:57.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:06:57.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:06:57.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:06:57.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:06:57.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:06:57.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:06:57.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:06:57.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:06:57.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:06:57.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:06:57.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:06:57.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:06:57.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:06:57.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:06:57.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:06:57.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:06:57.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:06:57.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:06:57.551 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:06:57.551 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:06:57.551 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:06:57.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:57.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:06:57.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:06:57.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:06:57.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:06:57.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:06:57.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:06:57.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:06:57.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:06:57.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:06:57.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:06:57.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:06:57.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:06:57.556 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:06:57.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:06:57.556 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:06:57.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:06:57.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:06:57.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:06:57.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:06:57.570 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:57.568+0000 7fa6773b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:57.571 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:57.572+0000 7fa6773b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:57.572 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:57.572+0000 7fa6773b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:06:57.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:06:57.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:06:57.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:06:57.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:06:57.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:06:57.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:06:57.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:06:57.715 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:06:57.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:06:57.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:06:57.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:06:57.996 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:57.996+0000 7fa6773b2a40 -1 Falling back to public interface 2026-02-19T13:06:58.630 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:06:58.628+0000 7fa6773b2a40 -1 osd.1 83 log_to_monitors true 2026-02-19T13:06:58.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:06:58.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:06:58.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:06:58.871 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:06:58.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:06:58.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:06:59.029 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 87 up_thru 87 down_at 84 last_clean_interval [77,83) [v2:127.0.0.1:6810/644486589,v1:127.0.0.1:6811/644486589] [v2:127.0.0.1:6812/644486589,v1:127.0.0.1:6813/644486589] exists,up c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:06:59.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:06:59.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:06:59.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:06:59.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:06:59.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:06:59.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:06:59.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:06:59.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:06:59.031 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:06:59.031 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:06:59.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:06:59.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:06:59.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:06:59.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:06:59.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:06:59.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:06:59.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:06:59.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:06:59.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:06:59.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:06:59.303 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:06:59.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:06:59.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:59.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:06:59.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=352187318280 2026-02-19T13:06:59.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 352187318280 2026-02-19T13:06:59.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-352187318280' 2026-02-19T13:06:59.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:06:59.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:06:59.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=373662154754 2026-02-19T13:06:59.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 373662154754 2026-02-19T13:06:59.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-352187318280 1-373662154754' 2026-02-19T13:06:59.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:06:59.452 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-352187318280 2026-02-19T13:06:59.452 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:06:59.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:06:59.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-352187318280 2026-02-19T13:06:59.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:06:59.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=352187318280 2026-02-19T13:06:59.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 352187318280' 2026-02-19T13:06:59.456 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 352187318280 2026-02-19T13:06:59.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:06:59.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 352187318278 -lt 352187318280 2026-02-19T13:06:59.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:00.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:07:00.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:00.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 352187318278 -lt 352187318280 2026-02-19T13:07:00.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:01.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:07:01.788 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:01.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 352187318281 -lt 352187318280 2026-02-19T13:07:01.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:01.953 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-373662154754 2026-02-19T13:07:01.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:01.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:07:01.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-373662154754 2026-02-19T13:07:01.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:01.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=373662154754 2026-02-19T13:07:01.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 373662154754' 2026-02-19T13:07:01.958 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 373662154754 2026-02-19T13:07:01.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:07:02.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 373662154756 -lt 373662154754 2026-02-19T13:07:02.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:07:02.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:02.126 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:02.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:07:02.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:07:02.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:07:02.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:07:02.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:07:02.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:07:02.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:07:02.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:07:02.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:07:02.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:07:02.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:02.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:02.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:07:02.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:07:02.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:07:02.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:07:02.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ6 2026-02-19T13:07:02.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 6 % 2 2026-02-19T13:07:02.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=0 2026-02-19T13:07:02.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:07:02.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6518: TEST_dual_store_replicated_cluster: echo extra 2026-02-19T13:07:02.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6519: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-19T13:07:02.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:07:02.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:07:02.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:07:02.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:07:02.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-19T13:07:02.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:07:02.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:07:02.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:07:02.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:07:02.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:07:02.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:07:02.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:07:02.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:07:02.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:07:02.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:07:02.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:07:02.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-19T13:07:02.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:07:02.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:07:02.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:07:02.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:07:02.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:07:02.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-19T13:07:03.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:03.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:07:03.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:07:03.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:07:03.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:07:03.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:07:03.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:07:03.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:07:03.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:07:03.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:07:03.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:07:03.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:07:03.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:07:03.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:07:03.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:07:03.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:07:03.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:07:03.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:07:03.247 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:07:03.247 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:07:03.247 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:07:03.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:03.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:03.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:07:03.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:07:03.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:07:03.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:07:03.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:07:03.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:07:03.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:07:03.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:07:03.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:07:03.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:07:03.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:07:03.250 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:07:03.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:03.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:07:03.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:07:03.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:07:03.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:07:03.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:07:03.265 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:03.263+0000 7feb99886a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:03.267 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:03.267+0000 7feb99886a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:03.268 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:03.267+0000 7feb99886a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:03.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:07:03.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:07:03.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:07:03.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:07:03.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:07:03.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:03.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:07:03.409 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:07:03.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:03.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:07:03.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:07:03.680 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:03.679+0000 7feb99886a40 -1 Falling back to public interface 2026-02-19T13:07:04.256 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:04.255+0000 7feb99886a40 -1 osd.0 88 log_to_monitors true 2026-02-19T13:07:04.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:07:04.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:04.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:07:04.571 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:07:04.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:04.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:07:04.731 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 92 up_thru 92 down_at 89 last_clean_interval [82,88) [v2:127.0.0.1:6802/1562561212,v1:127.0.0.1:6803/1562561212] [v2:127.0.0.1:6804/1562561212,v1:127.0.0.1:6805/1562561212] exists,up fe651040-8a56-4e0e-9bdd-37b3fbeea15d 2026-02-19T13:07:04.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:07:04.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:07:04.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:07:04.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:07:04.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:07:04.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:07:04.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:07:04.732 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:07:04.733 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:07:04.733 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:07:04.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:07:04.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:07:04.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:07:04.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:07:04.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:07:04.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:07:04.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:07:04.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:07:04.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:07:05.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:07:05.000 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:07:05.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:07:05.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:05.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:07:05.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=395136991234 2026-02-19T13:07:05.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 395136991234 2026-02-19T13:07:05.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-395136991234' 2026-02-19T13:07:05.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:05.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:07:05.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=373662154761 2026-02-19T13:07:05.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 373662154761 2026-02-19T13:07:05.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-395136991234 1-373662154761' 2026-02-19T13:07:05.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:05.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-395136991234 2026-02-19T13:07:05.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:05.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:07:05.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-395136991234 2026-02-19T13:07:05.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:05.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=395136991234 2026-02-19T13:07:05.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 395136991234' 2026-02-19T13:07:05.151 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 395136991234 2026-02-19T13:07:05.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:05.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 395136991234 -lt 395136991234 2026-02-19T13:07:05.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:05.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-373662154761 2026-02-19T13:07:05.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:05.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:07:05.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-373662154761 2026-02-19T13:07:05.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:05.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=373662154761 2026-02-19T13:07:05.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 373662154761' 2026-02-19T13:07:05.313 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 373662154761 2026-02-19T13:07:05.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:07:05.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 373662154760 -lt 373662154761 2026-02-19T13:07:05.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:06.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:07:06.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:07:06.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 373662154760 -lt 373662154761 2026-02-19T13:07:06.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:07.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:07:07.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:07:07.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 373662154763 -lt 373662154761 2026-02-19T13:07:07.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:07:07.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:07.811 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:07.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:07:07.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:07:07.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:07:07.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:07:07.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:07:07.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:07:07.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:07:07.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:07:08.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:07:08.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:07:08.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:08.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:08.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:07:08.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:07:08.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:07:08.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6520: TEST_dual_store_replicated_cluster: rm td/osd-scrub-repair/extra-val 2026-02-19T13:07:08.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:07:08.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ7 2026-02-19T13:07:08.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 7 % 2 2026-02-19T13:07:08.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=1 2026-02-19T13:07:08.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:07:08.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6525: TEST_dual_store_replicated_cluster: echo -n newheader 2026-02-19T13:07:08.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6526: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-19T13:07:08.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:07:08.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:07:08.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:07:08.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:07:08.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-19T13:07:08.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:07:08.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:07:08.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:07:08.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:07:08.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:07:08.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:07:08.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:07:08.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:07:08.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:07:08.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:07:08.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:07:08.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-19T13:07:08.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:07:08.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:07:08.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:07:08.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:07:08.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:08.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-19T13:07:08.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:08.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:07:08.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:07:08.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:07:08.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:07:08.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:08.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:07:08.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:07:08.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:07:08.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:07:08.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:07:08.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:07:08.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:07:08.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:07:08.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:07:08.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:07:08.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:07:08.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:07:08.974 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:07:08.974 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:07:08.975 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:07:08.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:08.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:08.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:07:08.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:07:08.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:07:08.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:07:08.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:07:08.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:07:08.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:07:08.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:07:08.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:07:08.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:07:08.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:07:08.977 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:07:08.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:08.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:07:08.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:07:08.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:07:08.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:07:08.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:07:08.992 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:08.991+0000 7f8900d92a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:08.993 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:08.991+0000 7f8900d92a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:08.994 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:08.991+0000 7f8900d92a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:09.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:07:09.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:07:09.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:07:09.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:07:09.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:07:09.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:09.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:07:09.138 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:07:09.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:09.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:09.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:07:09.372 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:09.371+0000 7f8900d92a40 -1 Falling back to public interface 2026-02-19T13:07:09.999 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:09.999+0000 7f8900d92a40 -1 osd.1 93 log_to_monitors true 2026-02-19T13:07:10.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:07:10.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:10.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:07:10.300 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:07:10.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:10.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:10.453 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 97 up_thru 97 down_at 94 last_clean_interval [87,93) [v2:127.0.0.1:6810/3434736689,v1:127.0.0.1:6811/3434736689] [v2:127.0.0.1:6812/3434736689,v1:127.0.0.1:6813/3434736689] exists,up c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:07:10.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:07:10.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:07:10.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:07:10.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:07:10.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:07:10.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:07:10.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:07:10.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:07:10.455 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:07:10.455 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:07:10.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:07:10.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:07:10.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:07:10.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:07:10.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:07:10.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:07:10.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:07:10.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:07:10.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:07:10.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:07:10.739 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:07:10.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:07:10.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:10.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:07:10.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=395136991241 2026-02-19T13:07:10.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 395136991241 2026-02-19T13:07:10.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-395136991241' 2026-02-19T13:07:10.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:10.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:07:10.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=416611827714 2026-02-19T13:07:10.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 416611827714 2026-02-19T13:07:10.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-395136991241 1-416611827714' 2026-02-19T13:07:10.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:10.890 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-395136991241 2026-02-19T13:07:10.890 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:10.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:07:10.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-395136991241 2026-02-19T13:07:10.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:10.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=395136991241 2026-02-19T13:07:10.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 395136991241' 2026-02-19T13:07:10.894 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 395136991241 2026-02-19T13:07:10.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:11.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 395136991238 -lt 395136991241 2026-02-19T13:07:11.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:12.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:07:12.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:12.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 395136991241 -lt 395136991241 2026-02-19T13:07:12.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:12.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-416611827714 2026-02-19T13:07:12.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:12.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:07:12.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-416611827714 2026-02-19T13:07:12.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:12.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=416611827714 2026-02-19T13:07:12.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 416611827714' 2026-02-19T13:07:12.229 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 416611827714 2026-02-19T13:07:12.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:07:12.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 416611827715 -lt 416611827714 2026-02-19T13:07:12.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:07:12.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:12.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:12.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:07:12.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:07:12.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:07:12.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:07:12.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:07:12.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:07:12.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:07:12.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:07:12.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:07:12.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:07:12.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:12.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:12.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:07:12.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:07:12.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:07:12.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6527: TEST_dual_store_replicated_cluster: rm td/osd-scrub-repair/hdr 2026-02-19T13:07:12.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:07:12.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ8 2026-02-19T13:07:12.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 8 % 2 2026-02-19T13:07:12.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=0 2026-02-19T13:07:12.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:07:12.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6531: TEST_dual_store_replicated_cluster: rados --pool csr_pool setxattr ROBJ8 key1-ROBJ8 val1-ROBJ8 2026-02-19T13:07:12.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6532: TEST_dual_store_replicated_cluster: rados --pool csr_pool setxattr ROBJ8 key2-ROBJ8 val2-ROBJ8 2026-02-19T13:07:12.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6535: TEST_dual_store_replicated_cluster: echo -n bad-val 2026-02-19T13:07:12.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6536: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-19T13:07:12.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:07:12.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:07:12.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:07:12.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:07:12.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-19T13:07:12.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:07:12.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:07:12.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:07:12.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:07:12.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:07:12.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:07:12.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:07:12.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:07:12.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:07:12.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:07:13.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:07:13.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-19T13:07:13.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:07:13.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:07:13.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:07:13.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:07:13.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:07:13.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-19T13:07:13.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:13.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:07:13.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:07:13.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:07:13.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:07:13.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:07:13.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:07:13.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:07:13.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:07:13.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:07:13.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:07:13.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:07:13.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:07:13.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:07:13.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:07:13.550 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:07:13.550 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:07:13.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:07:13.551 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:07:13.551 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:07:13.551 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:07:13.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:13.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:13.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:07:13.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:07:13.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:07:13.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:07:13.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:07:13.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:07:13.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:07:13.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:07:13.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:07:13.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:07:13.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:07:13.554 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:07:13.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:13.554 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:07:13.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:07:13.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:07:13.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:07:13.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:07:13.577 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:13.575+0000 7fdfe4adea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:13.578 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:13.575+0000 7fdfe4adea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:13.579 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:13.579+0000 7fdfe4adea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:13.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:07:13.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:07:13.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:07:13.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:07:13.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:07:13.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:13.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:07:13.708 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:07:13.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:13.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:07:13.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:07:13.936 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:13.935+0000 7fdfe4adea40 -1 Falling back to public interface 2026-02-19T13:07:14.514 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:14.511+0000 7fdfe4adea40 -1 osd.0 98 log_to_monitors true 2026-02-19T13:07:14.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:07:14.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:14.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:07:14.871 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:07:14.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:14.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:07:15.034 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 102 up_thru 102 down_at 99 last_clean_interval [92,98) [v2:127.0.0.1:6802/815711862,v1:127.0.0.1:6803/815711862] [v2:127.0.0.1:6804/815711862,v1:127.0.0.1:6805/815711862] exists,up fe651040-8a56-4e0e-9bdd-37b3fbeea15d 2026-02-19T13:07:15.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:07:15.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:07:15.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:07:15.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:07:15.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:07:15.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:07:15.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:07:15.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:07:15.036 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:07:15.036 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:07:15.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:07:15.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:07:15.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:07:15.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:07:15.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:07:15.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:07:15.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:07:15.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:07:15.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:07:15.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:07:15.318 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:07:15.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:07:15.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:15.318 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:07:15.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=438086664194 2026-02-19T13:07:15.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 438086664194 2026-02-19T13:07:15.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664194' 2026-02-19T13:07:15.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:15.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:07:15.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=416611827720 2026-02-19T13:07:15.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 416611827720 2026-02-19T13:07:15.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664194 1-416611827720' 2026-02-19T13:07:15.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:15.463 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-438086664194 2026-02-19T13:07:15.463 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:15.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:07:15.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-438086664194 2026-02-19T13:07:15.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:15.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=438086664194 2026-02-19T13:07:15.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 438086664194' 2026-02-19T13:07:15.467 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 438086664194 2026-02-19T13:07:15.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:15.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 438086664194 2026-02-19T13:07:15.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:16.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:07:16.636 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:16.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 438086664194 2026-02-19T13:07:16.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:17.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:07:17.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:17.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664196 -lt 438086664194 2026-02-19T13:07:17.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:17.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-416611827720 2026-02-19T13:07:17.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:17.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:07:17.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-416611827720 2026-02-19T13:07:17.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:17.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=416611827720 2026-02-19T13:07:17.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 416611827720' 2026-02-19T13:07:17.962 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 416611827720 2026-02-19T13:07:17.962 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:07:18.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 416611827722 -lt 416611827720 2026-02-19T13:07:18.131 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:07:18.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:18.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:18.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:07:18.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:07:18.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:07:18.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:07:18.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:07:18.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:07:18.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:07:18.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:07:18.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:07:18.488 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:07:18.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:18.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:18.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:07:18.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:07:18.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:07:18.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6537: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-19T13:07:18.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:07:18.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:07:18.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:07:18.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:07:18.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-19T13:07:18.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:07:18.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:07:18.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:07:18.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:07:18.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:07:18.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:07:18.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:07:18.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:07:18.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:07:18.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:07:18.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:07:18.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-19T13:07:18.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:07:18.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:07:18.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:07:18.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:07:18.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:18.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-19T13:07:19.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:19.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:07:19.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:07:19.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:07:19.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:07:19.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:19.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:07:19.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:07:19.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:07:19.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:07:19.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:07:19.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:07:19.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:07:19.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:07:19.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:07:19.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:07:19.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:07:19.287 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:07:19.287 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:07:19.287 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:07:19.287 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:07:19.287 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:19.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:19.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:07:19.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:07:19.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:07:19.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:07:19.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:07:19.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:07:19.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:07:19.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:07:19.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:07:19.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:07:19.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:07:19.290 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:07:19.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:19.290 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:07:19.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:07:19.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:07:19.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:07:19.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:07:19.305 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:19.303+0000 7fe69859ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:19.307 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:19.307+0000 7fe69859ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:19.308 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:19.307+0000 7fe69859ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:19.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:07:19.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:07:19.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:07:19.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:07:19.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:07:19.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:19.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:07:19.450 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:07:19.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:19.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:19.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:07:19.728 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:19.727+0000 7fe69859ca40 -1 Falling back to public interface 2026-02-19T13:07:20.374 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:20.371+0000 7fe69859ca40 -1 osd.1 104 log_to_monitors true 2026-02-19T13:07:20.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:07:20.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:20.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:07:20.615 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:07:20.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:20.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:20.770 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 108 up_thru 108 down_at 105 last_clean_interval [97,104) [v2:127.0.0.1:6810/60438609,v1:127.0.0.1:6811/60438609] [v2:127.0.0.1:6812/60438609,v1:127.0.0.1:6813/60438609] exists,up c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:07:20.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:07:20.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:07:20.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:07:20.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:07:20.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:07:20.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:07:20.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:07:20.771 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:07:20.772 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:07:20.773 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:07:20.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:07:20.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:07:20.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:07:20.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:07:20.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:07:20.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:07:20.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:07:20.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:07:20.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:07:21.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:07:21.052 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:07:21.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:07:21.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:21.052 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:07:21.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=438086664201 2026-02-19T13:07:21.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 438086664201 2026-02-19T13:07:21.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664201' 2026-02-19T13:07:21.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:21.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:07:21.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=463856467970 2026-02-19T13:07:21.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 463856467970 2026-02-19T13:07:21.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664201 1-463856467970' 2026-02-19T13:07:21.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:21.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-438086664201 2026-02-19T13:07:21.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:21.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:07:21.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-438086664201 2026-02-19T13:07:21.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:21.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=438086664201 2026-02-19T13:07:21.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 438086664201' 2026-02-19T13:07:21.203 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 438086664201 2026-02-19T13:07:21.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:21.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664200 -lt 438086664201 2026-02-19T13:07:21.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:22.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:07:22.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:22.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664200 -lt 438086664201 2026-02-19T13:07:22.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:23.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:07:23.536 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:23.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664203 -lt 438086664201 2026-02-19T13:07:23.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:23.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-463856467970 2026-02-19T13:07:23.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:23.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:07:23.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-463856467970 2026-02-19T13:07:23.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:23.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=463856467970 2026-02-19T13:07:23.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 463856467970' 2026-02-19T13:07:23.701 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 463856467970 2026-02-19T13:07:23.701 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:07:23.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 463856467972 -lt 463856467970 2026-02-19T13:07:23.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:07:23.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:23.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:24.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:07:24.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:07:24.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:07:24.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:07:24.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:07:24.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:07:24.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:07:24.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:07:24.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:07:24.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:07:24.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:24.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:24.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:07:24.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:07:24.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:07:24.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6538: TEST_dual_store_replicated_cluster: echo -n val3-ROBJ8 2026-02-19T13:07:24.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6539: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-19T13:07:24.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:07:24.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:07:24.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:07:24.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:07:24.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-19T13:07:24.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:07:24.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:07:24.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:07:24.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:07:24.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:07:24.420 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:07:24.420 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:07:24.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:07:24.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:07:24.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:07:24.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:07:24.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-19T13:07:24.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:07:24.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:07:24.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:07:24.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:07:24.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:24.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-19T13:07:25.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:25.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:07:25.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:07:25.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:07:25.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:07:25.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:25.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:07:25.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:07:25.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:07:25.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:07:25.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:07:25.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:07:25.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:07:25.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:07:25.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:07:25.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:07:25.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:07:25.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:07:25.043 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:07:25.043 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:07:25.043 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:07:25.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:25.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:25.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:07:25.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:07:25.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:07:25.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:07:25.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:07:25.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:07:25.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:07:25.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:07:25.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:07:25.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:07:25.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:07:25.046 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:07:25.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:25.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:07:25.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:07:25.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:07:25.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:07:25.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:07:25.060 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:25.059+0000 7f01de2aea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:25.061 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:25.059+0000 7f01de2aea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:25.062 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:25.059+0000 7f01de2aea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:25.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:07:25.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:07:25.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:07:25.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:07:25.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:07:25.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:25.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:07:25.204 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:07:25.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:25.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:25.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:07:25.504 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:25.503+0000 7f01de2aea40 -1 Falling back to public interface 2026-02-19T13:07:26.084 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:26.083+0000 7f01de2aea40 -1 osd.1 109 log_to_monitors true 2026-02-19T13:07:26.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:07:26.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:26.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:07:26.361 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:07:26.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:26.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:26.513 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 113 up_thru 113 down_at 110 last_clean_interval [108,109) [v2:127.0.0.1:6810/4115402190,v1:127.0.0.1:6811/4115402190] [v2:127.0.0.1:6812/4115402190,v1:127.0.0.1:6813/4115402190] exists,up c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:07:26.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:07:26.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:07:26.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:07:26.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:07:26.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:07:26.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:07:26.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:07:26.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:07:26.515 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:07:26.515 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:07:26.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:07:26.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:07:26.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:07:26.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:07:26.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:07:26.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:07:26.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:07:26.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:07:26.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:07:26.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:07:26.802 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:07:26.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:07:26.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:26.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:07:26.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=438086664208 2026-02-19T13:07:26.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 438086664208 2026-02-19T13:07:26.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664208' 2026-02-19T13:07:26.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:26.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:07:26.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=485331304450 2026-02-19T13:07:26.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 485331304450 2026-02-19T13:07:26.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664208 1-485331304450' 2026-02-19T13:07:26.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:26.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-438086664208 2026-02-19T13:07:26.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:26.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:07:26.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-438086664208 2026-02-19T13:07:26.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:26.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=438086664208 2026-02-19T13:07:26.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 438086664208' 2026-02-19T13:07:26.950 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 438086664208 2026-02-19T13:07:26.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:27.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664205 -lt 438086664208 2026-02-19T13:07:27.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:28.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:07:28.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:28.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664208 -lt 438086664208 2026-02-19T13:07:28.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:28.279 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-485331304450 2026-02-19T13:07:28.279 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:28.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:07:28.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-485331304450 2026-02-19T13:07:28.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:28.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=485331304450 2026-02-19T13:07:28.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 485331304450' 2026-02-19T13:07:28.284 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 485331304450 2026-02-19T13:07:28.284 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:07:28.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 485331304450 -lt 485331304450 2026-02-19T13:07:28.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:07:28.443 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:28.443 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:28.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:07:28.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:07:28.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:07:28.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:07:28.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:07:28.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:07:28.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:07:28.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:07:28.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:07:28.785 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:07:28.785 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:28.785 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:28.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:07:28.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:07:28.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:07:28.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6540: TEST_dual_store_replicated_cluster: rm td/osd-scrub-repair/bad-val td/osd-scrub-repair/newval 2026-02-19T13:07:28.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:07:28.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ9 2026-02-19T13:07:28.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 9 % 2 2026-02-19T13:07:28.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=1 2026-02-19T13:07:28.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:07:28.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6544: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 1 ROBJ9 get-attr _ 2026-02-19T13:07:28.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:07:28.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:07:28.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:07:28.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:07:28.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ9 get-attr _ 2026-02-19T13:07:28.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:07:28.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:07:28.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:07:28.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:07:28.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:07:28.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:07:28.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:07:28.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:07:28.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:07:28.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:07:29.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:07:29.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ9 get-attr _ 2026-02-19T13:07:29.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:07:29.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:07:29.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:07:29.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:07:29.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:29.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ9 get-attr _ 2026-02-19T13:07:29.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:29.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:07:29.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:07:29.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:07:29.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:07:29.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:29.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:07:29.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:07:29.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:07:29.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:07:29.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:07:29.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:07:29.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:07:29.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:07:29.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:07:29.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:07:29.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:07:29.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:07:29.462 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:07:29.462 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:07:29.462 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:07:29.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:29.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:29.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:07:29.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:07:29.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:07:29.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:07:29.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:07:29.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:07:29.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:07:29.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:07:29.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:07:29.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:07:29.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:07:29.463 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:07:29.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:29.463 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:07:29.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:07:29.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:07:29.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:07:29.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:07:29.483 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:29.483+0000 7f126dbf3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:29.485 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:29.483+0000 7f126dbf3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:29.486 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:29.483+0000 7f126dbf3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:29.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:07:29.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:07:29.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:07:29.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:07:29.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:07:29.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:29.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:07:29.618 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:07:29.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:29.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:29.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:07:29.896 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:29.895+0000 7f126dbf3a40 -1 Falling back to public interface 2026-02-19T13:07:30.470 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:30.467+0000 7f126dbf3a40 -1 osd.1 115 log_to_monitors true 2026-02-19T13:07:30.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:07:30.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:30.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:07:30.779 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:07:30.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:30.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:30.938 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 119 up_thru 119 down_at 116 last_clean_interval [113,115) [v2:127.0.0.1:6810/769701473,v1:127.0.0.1:6811/769701473] [v2:127.0.0.1:6812/769701473,v1:127.0.0.1:6813/769701473] exists,up c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:07:30.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:07:30.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:07:30.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:07:30.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:07:30.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:07:30.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:07:30.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:07:30.939 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:07:30.939 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:07:30.939 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:07:30.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:07:30.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:07:30.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:07:31.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:07:31.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:07:31.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:07:31.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:07:31.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:07:31.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:07:31.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:07:31.214 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:07:31.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:07:31.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:31.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:07:31.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=438086664213 2026-02-19T13:07:31.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 438086664213 2026-02-19T13:07:31.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664213' 2026-02-19T13:07:31.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:31.289 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:07:31.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=511101108226 2026-02-19T13:07:31.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 511101108226 2026-02-19T13:07:31.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664213 1-511101108226' 2026-02-19T13:07:31.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:31.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-438086664213 2026-02-19T13:07:31.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:31.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:07:31.365 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-438086664213 2026-02-19T13:07:31.365 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:31.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=438086664213 2026-02-19T13:07:31.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 438086664213' 2026-02-19T13:07:31.367 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 438086664213 2026-02-19T13:07:31.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:31.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664212 -lt 438086664213 2026-02-19T13:07:31.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:32.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:07:32.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:32.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664212 -lt 438086664213 2026-02-19T13:07:32.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:33.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:07:33.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:33.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664215 -lt 438086664213 2026-02-19T13:07:33.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:33.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-511101108226 2026-02-19T13:07:33.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:33.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:07:33.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-511101108226 2026-02-19T13:07:33.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:33.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=511101108226 2026-02-19T13:07:33.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 511101108226' 2026-02-19T13:07:33.855 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 511101108226 2026-02-19T13:07:33.856 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:07:34.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 511101108228 -lt 511101108226 2026-02-19T13:07:34.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:07:34.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:34.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:34.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:07:34.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:07:34.205 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:07:34.205 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:07:34.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:07:34.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:07:34.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:07:34.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:07:34.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:07:34.363 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:07:34.363 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:34.364 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:34.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:07:34.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:07:34.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:07:34.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6545: TEST_dual_store_replicated_cluster: echo -n D 2026-02-19T13:07:34.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6546: TEST_dual_store_replicated_cluster: rados --pool csr_pool put ROBJ9 td/osd-scrub-repair/change 2026-02-19T13:07:34.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6547: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-19T13:07:34.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:07:34.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:07:34.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:07:34.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:07:34.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-19T13:07:34.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:07:34.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:07:34.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:07:34.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:07:34.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:07:34.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:07:34.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:07:34.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:07:34.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:07:34.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:07:34.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:07:34.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-19T13:07:34.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:07:34.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:07:34.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:07:34.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:07:34.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:34.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-19T13:07:35.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:35.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:07:35.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:07:35.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:07:35.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:07:35.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:35.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:07:35.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:07:35.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:07:35.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:07:35.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:07:35.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:07:35.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:07:35.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:07:35.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:07:35.182 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:07:35.182 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:07:35.182 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:07:35.182 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:07:35.182 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:07:35.182 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:07:35.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:35.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:35.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:07:35.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:07:35.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:07:35.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:07:35.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:07:35.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:07:35.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:07:35.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:07:35.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:07:35.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:07:35.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:07:35.185 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:07:35.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:35.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:07:35.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:07:35.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:07:35.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:07:35.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:07:35.201 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:35.199+0000 7f4db392ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:35.203 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:35.203+0000 7f4db392ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:35.204 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:35.203+0000 7f4db392ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:35.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:07:35.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:07:35.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:07:35.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:07:35.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:07:35.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:35.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:07:35.347 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:07:35.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:35.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:35.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:07:35.576 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:35.575+0000 7f4db392ca40 -1 Falling back to public interface 2026-02-19T13:07:36.235 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:36.235+0000 7f4db392ca40 -1 osd.1 120 log_to_monitors true 2026-02-19T13:07:36.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:07:36.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:36.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:07:36.505 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:07:36.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:36.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:36.659 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 124 up_thru 124 down_at 121 last_clean_interval [119,120) [v2:127.0.0.1:6810/1554389254,v1:127.0.0.1:6811/1554389254] [v2:127.0.0.1:6812/1554389254,v1:127.0.0.1:6813/1554389254] exists,up c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:07:36.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:07:36.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:07:36.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:07:36.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:07:36.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:07:36.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:07:36.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:07:36.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:07:36.661 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:07:36.661 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:07:36.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:07:36.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:07:36.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:07:36.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:07:36.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:07:36.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:07:36.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:07:36.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:07:36.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:07:36.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:07:36.938 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:07:36.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:07:36.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:36.939 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:07:37.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=438086664220 2026-02-19T13:07:37.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 438086664220 2026-02-19T13:07:37.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664220' 2026-02-19T13:07:37.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:37.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:07:37.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=532575944706 2026-02-19T13:07:37.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 532575944706 2026-02-19T13:07:37.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-438086664220 1-532575944706' 2026-02-19T13:07:37.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:37.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-438086664220 2026-02-19T13:07:37.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:37.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:07:37.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-438086664220 2026-02-19T13:07:37.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:37.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=438086664220 2026-02-19T13:07:37.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 438086664220' 2026-02-19T13:07:37.090 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 438086664220 2026-02-19T13:07:37.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:37.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664220 -lt 438086664220 2026-02-19T13:07:37.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:37.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-532575944706 2026-02-19T13:07:37.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:37.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:07:37.249 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-532575944706 2026-02-19T13:07:37.249 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:37.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=532575944706 2026-02-19T13:07:37.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 532575944706' 2026-02-19T13:07:37.251 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 532575944706 2026-02-19T13:07:37.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:07:37.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 532575944706 -lt 532575944706 2026-02-19T13:07:37.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:07:37.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:37.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:37.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:07:37.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:07:37.606 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:07:37.606 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:07:37.606 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:07:37.606 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:07:37.606 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:07:37.606 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:07:37.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:07:37.768 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:07:37.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:37.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:37.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:07:37.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:07:37.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:07:37.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6548: TEST_dual_store_replicated_cluster: rm td/osd-scrub-repair/oi td/osd-scrub-repair/change 2026-02-19T13:07:37.964 INFO:tasks.workunit.client.0.trial144.stderr:rm: cannot remove 'td/osd-scrub-repair/oi': No such file or directory 2026-02-19T13:07:37.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:07:37.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ10 2026-02-19T13:07:37.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 10 % 2 2026-02-19T13:07:37.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=0 2026-02-19T13:07:37.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:07:37.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:07:37.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ11 2026-02-19T13:07:37.966 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 11 % 2 2026-02-19T13:07:37.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=1 2026-02-19T13:07:37.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:07:37.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:07:37.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ12 2026-02-19T13:07:37.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 12 % 2 2026-02-19T13:07:37.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=0 2026-02-19T13:07:37.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:07:37.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:07:37.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ13 2026-02-19T13:07:37.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 13 % 2 2026-02-19T13:07:37.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=1 2026-02-19T13:07:37.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:07:37.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:07:37.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ14 2026-02-19T13:07:37.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 14 % 2 2026-02-19T13:07:37.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=0 2026-02-19T13:07:37.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:07:37.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6557: TEST_dual_store_replicated_cluster: echo -n bad-val 2026-02-19T13:07:37.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6558: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-19T13:07:37.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:07:37.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:07:37.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:07:37.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:07:37.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-19T13:07:37.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:07:37.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:07:37.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:07:37.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:07:37.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:07:37.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:07:37.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:07:37.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:07:37.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:07:37.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:07:38.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:07:38.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-19T13:07:38.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:07:38.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:07:38.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:07:38.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:07:38.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:07:38.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-19T13:07:38.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:38.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:07:38.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:07:38.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:07:38.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:07:38.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:07:38.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:07:38.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:07:38.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:07:38.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:07:38.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:07:38.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:07:38.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:07:38.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:07:38.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:07:38.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:07:38.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:07:38.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:07:38.582 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:07:38.582 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:07:38.582 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:07:38.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:38.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:38.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:07:38.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:07:38.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:07:38.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:07:38.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:07:38.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:07:38.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:07:38.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:07:38.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:07:38.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:07:38.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:07:38.583 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:07:38.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:38.583 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:07:38.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:07:38.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:07:38.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:07:38.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:07:38.604 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:38.603+0000 7fdfacf6ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:38.606 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:38.603+0000 7fdfacf6ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:38.607 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:38.607+0000 7fdfacf6ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:38.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:07:38.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:07:38.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:07:38.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:07:38.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:07:38.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:38.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:07:38.746 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:07:38.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:38.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:07:38.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:07:38.996 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:38.995+0000 7fdfacf6ea40 -1 Falling back to public interface 2026-02-19T13:07:39.598 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:39.595+0000 7fdfacf6ea40 -1 osd.0 126 log_to_monitors true 2026-02-19T13:07:39.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:07:39.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:39.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:07:39.904 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:07:39.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:39.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:07:40.057 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 130 up_thru 130 down_at 127 last_clean_interval [102,126) [v2:127.0.0.1:6802/1549458175,v1:127.0.0.1:6803/1549458175] [v2:127.0.0.1:6804/1549458175,v1:127.0.0.1:6805/1549458175] exists,up fe651040-8a56-4e0e-9bdd-37b3fbeea15d 2026-02-19T13:07:40.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:07:40.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:07:40.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:07:40.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:07:40.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:07:40.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:07:40.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:07:40.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:07:40.059 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:07:40.059 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:07:40.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:07:40.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:07:40.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:07:40.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:07:40.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:07:40.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:07:40.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:07:40.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:07:40.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:07:40.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:07:40.326 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:07:40.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:07:40.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:40.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:07:40.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=558345748482 2026-02-19T13:07:40.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 558345748482 2026-02-19T13:07:40.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-558345748482' 2026-02-19T13:07:40.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:40.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:07:40.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=532575944711 2026-02-19T13:07:40.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 532575944711 2026-02-19T13:07:40.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-558345748482 1-532575944711' 2026-02-19T13:07:40.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:40.472 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-558345748482 2026-02-19T13:07:40.472 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:40.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:07:40.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-558345748482 2026-02-19T13:07:40.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:40.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=558345748482 2026-02-19T13:07:40.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 558345748482' 2026-02-19T13:07:40.476 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 558345748482 2026-02-19T13:07:40.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:40.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 438086664221 -lt 558345748482 2026-02-19T13:07:40.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:41.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:07:41.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:41.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 558345748483 -lt 558345748482 2026-02-19T13:07:41.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:41.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-532575944711 2026-02-19T13:07:41.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:41.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:07:41.800 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-532575944711 2026-02-19T13:07:41.800 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:41.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=532575944711 2026-02-19T13:07:41.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 532575944711' 2026-02-19T13:07:41.802 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 532575944711 2026-02-19T13:07:41.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:07:41.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 532575944711 -lt 532575944711 2026-02-19T13:07:41.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:07:41.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:41.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:42.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:07:42.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:07:42.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:07:42.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:07:42.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:07:42.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:07:42.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:07:42.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:07:42.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:07:42.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:07:42.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:42.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:42.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:07:42.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:07:42.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:07:42.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6559: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 1 ROBJ14 rm-attr _ 2026-02-19T13:07:42.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:07:42.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:07:42.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:07:42.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:07:42.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ14 rm-attr _ 2026-02-19T13:07:42.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:07:42.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:07:42.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:07:42.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:07:42.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:07:42.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:07:42.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:07:42.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:07:42.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:07:42.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:07:42.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:07:42.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ14 rm-attr _ 2026-02-19T13:07:42.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:07:42.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:07:42.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:07:42.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:07:42.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:42.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ14 rm-attr _ 2026-02-19T13:07:43.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:43.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:07:43.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:07:43.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:07:43.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:07:43.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:43.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:07:43.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:07:43.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:07:43.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:07:43.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:07:43.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:07:43.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:07:43.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:07:43.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:07:43.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:07:43.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:07:43.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:07:43.108 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:07:43.108 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:07:43.108 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:07:43.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:43.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:43.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:07:43.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:07:43.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:07:43.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:07:43.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:07:43.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:07:43.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:07:43.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:07:43.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:07:43.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:07:43.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:07:43.110 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:07:43.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:43.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:07:43.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:07:43.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:07:43.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:07:43.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:07:43.132 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:43.131+0000 7f492b48ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:43.133 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:43.131+0000 7f492b48ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:43.134 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:43.131+0000 7f492b48ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:43.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:07:43.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:07:43.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:07:43.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:07:43.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:07:43.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:43.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:07:43.266 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:07:43.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:43.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:43.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:07:43.548 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:43.547+0000 7f492b48ea40 -1 Falling back to public interface 2026-02-19T13:07:44.191 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:44.191+0000 7f492b48ea40 -1 osd.1 131 log_to_monitors true 2026-02-19T13:07:44.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:07:44.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:44.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:07:44.426 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:07:44.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:44.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:44.586 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 135 up_thru 135 down_at 132 last_clean_interval [124,131) [v2:127.0.0.1:6810/162479147,v1:127.0.0.1:6811/162479147] [v2:127.0.0.1:6812/162479147,v1:127.0.0.1:6813/162479147] exists,up c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:07:44.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:07:44.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:07:44.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:07:44.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:07:44.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:07:44.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:07:44.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:07:44.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:07:44.588 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:07:44.588 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:07:44.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:07:44.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:07:44.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:07:44.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:07:44.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:07:44.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:07:44.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:07:44.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:07:44.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:07:44.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:07:44.868 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:07:44.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:07:44.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:44.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:07:44.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=558345748488 2026-02-19T13:07:44.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 558345748488 2026-02-19T13:07:44.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-558345748488' 2026-02-19T13:07:44.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:44.942 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:07:45.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=579820584962 2026-02-19T13:07:45.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 579820584962 2026-02-19T13:07:45.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-558345748488 1-579820584962' 2026-02-19T13:07:45.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:45.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-558345748488 2026-02-19T13:07:45.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:45.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:07:45.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-558345748488 2026-02-19T13:07:45.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:45.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=558345748488 2026-02-19T13:07:45.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 558345748488' 2026-02-19T13:07:45.020 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 558345748488 2026-02-19T13:07:45.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:45.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 558345748488 -lt 558345748488 2026-02-19T13:07:45.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:45.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-579820584962 2026-02-19T13:07:45.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:45.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:07:45.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-579820584962 2026-02-19T13:07:45.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:45.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=579820584962 2026-02-19T13:07:45.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 579820584962' 2026-02-19T13:07:45.192 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 579820584962 2026-02-19T13:07:45.193 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:07:45.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 579820584962 -lt 579820584962 2026-02-19T13:07:45.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:07:45.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:45.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:45.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:07:45.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:07:45.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:07:45.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:07:45.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:07:45.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:07:45.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:07:45.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:07:45.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:07:45.702 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:07:45.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:45.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:45.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:07:45.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:07:45.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:07:45.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6560: TEST_dual_store_replicated_cluster: rm td/osd-scrub-repair/bad-val 2026-02-19T13:07:45.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:07:45.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ15 2026-02-19T13:07:45.893 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 15 % 2 2026-02-19T13:07:45.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=1 2026-02-19T13:07:45.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:07:45.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6564: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 1 ROBJ15 rm-attr _ 2026-02-19T13:07:45.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:07:45.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:07:45.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:07:45.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:07:45.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ15 rm-attr _ 2026-02-19T13:07:45.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:07:45.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:07:45.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:07:45.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:07:45.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:07:45.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:07:45.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:07:45.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:07:45.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:07:45.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:07:46.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:07:46.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ15 rm-attr _ 2026-02-19T13:07:46.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:07:46.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:07:46.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:07:46.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:07:46.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:46.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ15 rm-attr _ 2026-02-19T13:07:46.395 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-19T13:07:46.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:46.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:07:46.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:07:46.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:07:46.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:07:46.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:46.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:07:46.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:07:46.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:07:46.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:07:46.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:07:46.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:07:46.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:07:46.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:07:46.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:07:46.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:07:46.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:07:46.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:07:46.486 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:07:46.486 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:07:46.486 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:07:46.487 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:46.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:46.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:07:46.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:07:46.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:07:46.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:07:46.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:07:46.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:07:46.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:07:46.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:07:46.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:07:46.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:07:46.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:07:46.489 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:07:46.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:46.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:07:46.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:07:46.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:07:46.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:07:46.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:07:46.506 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:46.502+0000 7fd005c77a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:46.508 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:46.506+0000 7fd005c77a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:46.509 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:46.506+0000 7fd005c77a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:46.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:07:46.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:07:46.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:07:46.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:07:46.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:07:46.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:46.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:07:46.647 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:07:46.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:46.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:46.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:07:46.892 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:46.890+0000 7fd005c77a40 -1 Falling back to public interface 2026-02-19T13:07:47.524 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:47.522+0000 7fd005c77a40 -1 osd.1 137 log_to_monitors true 2026-02-19T13:07:47.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:07:47.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:47.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:07:47.808 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:07:47.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:47.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:47.967 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 141 up_thru 141 down_at 138 last_clean_interval [135,137) [v2:127.0.0.1:6810/3068094429,v1:127.0.0.1:6811/3068094429] [v2:127.0.0.1:6812/3068094429,v1:127.0.0.1:6813/3068094429] exists,up c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:07:47.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:07:47.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:07:47.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:07:47.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:07:47.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:07:47.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:07:47.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:07:47.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:07:47.969 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:07:47.969 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:07:47.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:07:47.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:07:47.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:07:48.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:07:48.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:07:48.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:07:48.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:07:48.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:07:48.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:07:48.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:07:48.244 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:07:48.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:07:48.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:48.245 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:07:48.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=558345748492 2026-02-19T13:07:48.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 558345748492 2026-02-19T13:07:48.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-558345748492' 2026-02-19T13:07:48.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:48.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:07:48.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=605590388738 2026-02-19T13:07:48.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 605590388738 2026-02-19T13:07:48.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-558345748492 1-605590388738' 2026-02-19T13:07:48.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:48.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-558345748492 2026-02-19T13:07:48.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:48.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:07:48.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-558345748492 2026-02-19T13:07:48.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:48.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=558345748492 2026-02-19T13:07:48.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 558345748492' 2026-02-19T13:07:48.395 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 558345748492 2026-02-19T13:07:48.396 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:48.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 558345748490 -lt 558345748492 2026-02-19T13:07:48.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:49.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:07:49.556 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:49.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 558345748493 -lt 558345748492 2026-02-19T13:07:49.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:49.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-605590388738 2026-02-19T13:07:49.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:49.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:07:49.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-605590388738 2026-02-19T13:07:49.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:49.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=605590388738 2026-02-19T13:07:49.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 605590388738' 2026-02-19T13:07:49.720 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 605590388738 2026-02-19T13:07:49.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:07:49.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 605590388739 -lt 605590388738 2026-02-19T13:07:49.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:07:49.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:49.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:50.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:07:50.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:07:50.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:07:50.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:07:50.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:07:50.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:07:50.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:07:50.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:07:50.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:07:50.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:07:50.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:50.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:50.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:07:50.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:07:50.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:07:50.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:07:50.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ16 2026-02-19T13:07:50.417 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 16 % 2 2026-02-19T13:07:50.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=0 2026-02-19T13:07:50.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:07:50.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6568: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 0 ROBJ16 rm-attr snapset 2026-02-19T13:07:50.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:07:50.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:07:50.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:07:50.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:07:50.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ16 rm-attr snapset 2026-02-19T13:07:50.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:07:50.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:07:50.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:07:50.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:07:50.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:07:50.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:07:50.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:07:50.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:07:50.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:07:50.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:07:50.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:07:50.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ16 rm-attr snapset 2026-02-19T13:07:50.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:07:50.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:07:50.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:07:50.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:07:50.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:07:50.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ16 rm-attr snapset 2026-02-19T13:07:50.891 INFO:tasks.workunit.client.0.trial144.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-19T13:07:51.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:51.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:07:51.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:07:51.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:07:51.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:07:51.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:07:51.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:07:51.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:07:51.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:07:51.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:07:51.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:07:51.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:07:51.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:07:51.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:07:51.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:07:51.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:07:51.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:07:51.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:07:51.009 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:07:51.009 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:07:51.009 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:07:51.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:51.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:51.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:07:51.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:07:51.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:07:51.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:07:51.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:07:51.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:07:51.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:07:51.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:07:51.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:07:51.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:07:51.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:07:51.011 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:07:51.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:51.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:07:51.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:07:51.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:07:51.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:07:51.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:07:51.028 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:51.026+0000 7f2209f38a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:51.030 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:51.026+0000 7f2209f38a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:51.031 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:51.030+0000 7f2209f38a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:51.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:07:51.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:07:51.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:07:51.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:07:51.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:07:51.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:51.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:07:51.168 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:07:51.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:51.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:07:51.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:07:51.425 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:51.422+0000 7f2209f38a40 -1 Falling back to public interface 2026-02-19T13:07:52.013 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:52.010+0000 7f2209f38a40 -1 osd.0 142 log_to_monitors true 2026-02-19T13:07:52.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:07:52.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:52.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:07:52.329 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:07:52.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:52.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:07:52.487 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 146 up_thru 146 down_at 143 last_clean_interval [130,142) [v2:127.0.0.1:6802/322743007,v1:127.0.0.1:6803/322743007] [v2:127.0.0.1:6804/322743007,v1:127.0.0.1:6805/322743007] exists,up fe651040-8a56-4e0e-9bdd-37b3fbeea15d 2026-02-19T13:07:52.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:07:52.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:07:52.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:07:52.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:07:52.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:07:52.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:07:52.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:07:52.488 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:07:52.489 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:07:52.489 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:07:52.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:07:52.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:07:52.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:07:52.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:07:52.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:07:52.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:07:52.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:07:52.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:07:52.607 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:07:52.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:07:52.762 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:07:52.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:07:52.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:52.762 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:07:52.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=627065225218 2026-02-19T13:07:52.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 627065225218 2026-02-19T13:07:52.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-627065225218' 2026-02-19T13:07:52.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:52.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:07:52.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=605590388744 2026-02-19T13:07:52.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 605590388744 2026-02-19T13:07:52.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-627065225218 1-605590388744' 2026-02-19T13:07:52.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:52.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-627065225218 2026-02-19T13:07:52.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:52.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:07:52.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-627065225218 2026-02-19T13:07:52.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:52.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=627065225218 2026-02-19T13:07:52.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 627065225218' 2026-02-19T13:07:52.910 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 627065225218 2026-02-19T13:07:52.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:53.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 558345748494 -lt 627065225218 2026-02-19T13:07:53.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:54.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:07:54.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:54.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 627065225218 -lt 627065225218 2026-02-19T13:07:54.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:54.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-605590388744 2026-02-19T13:07:54.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:54.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:07:54.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-605590388744 2026-02-19T13:07:54.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:54.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=605590388744 2026-02-19T13:07:54.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 605590388744' 2026-02-19T13:07:54.236 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 605590388744 2026-02-19T13:07:54.236 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:07:54.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 605590388744 -lt 605590388744 2026-02-19T13:07:54.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:07:54.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:54.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:54.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:07:54.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:07:54.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:07:54.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:07:54.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:07:54.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:07:54.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:07:54.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:07:54.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:07:54.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:07:54.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:07:54.752 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:07:54.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:07:54.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:07:54.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:07:54.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6569: TEST_dual_store_replicated_cluster: echo -n bad-val 2026-02-19T13:07:54.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6570: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-19T13:07:54.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:07:54.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:07:54.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:07:54.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:07:54.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-19T13:07:54.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:07:54.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:07:54.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:07:54.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:07:54.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:07:54.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:07:54.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:07:54.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:07:54.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:07:54.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:07:55.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:07:55.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-19T13:07:55.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:07:55.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:07:55.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:07:55.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:07:55.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:55.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-19T13:07:55.421 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-19T13:07:55.421 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-19T13:07:55.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:55.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:07:55.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:07:55.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:07:55.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:07:55.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:07:55.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:07:55.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:07:55.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:07:55.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:07:55.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:07:55.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:07:55.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:07:55.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:07:55.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:07:55.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:07:55.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:07:55.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:07:55.520 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:07:55.520 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:07:55.520 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:07:55.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:55.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:07:55.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:07:55.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:07:55.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:07:55.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:07:55.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:07:55.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:07:55.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:07:55.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:07:55.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:07:55.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:07:55.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:07:55.521 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:07:55.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:07:55.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:07:55.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:07:55.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:07:55.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:07:55.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:07:55.540 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:55.538+0000 7f1711f9ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:55.542 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:55.538+0000 7f1711f9ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:55.543 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:55.538+0000 7f1711f9ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:07:55.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:07:55.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:07:55.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:07:55.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:07:55.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:07:55.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:55.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:07:55.680 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:07:55.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:55.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:55.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:07:55.936 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:55.934+0000 7f1711f9ea40 -1 Falling back to public interface 2026-02-19T13:07:56.546 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:07:56.542+0000 7f1711f9ea40 -1 osd.1 147 log_to_monitors true 2026-02-19T13:07:56.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:07:56.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:07:56.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:07:56.840 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:07:56.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:07:56.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:07:56.996 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 151 up_thru 151 down_at 148 last_clean_interval [141,147) [v2:127.0.0.1:6810/762265947,v1:127.0.0.1:6811/762265947] [v2:127.0.0.1:6812/762265947,v1:127.0.0.1:6813/762265947] exists,up c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:07:56.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:07:56.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:07:56.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:07:56.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:07:56.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:07:56.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:07:56.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:07:56.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:07:56.998 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:07:56.998 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:07:56.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:07:56.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:07:56.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:07:57.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:07:57.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:07:57.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:07:57.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:07:57.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:07:57.120 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:07:57.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:07:57.280 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:07:57.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:07:57.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:57.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:07:57.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=627065225224 2026-02-19T13:07:57.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 627065225224 2026-02-19T13:07:57.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-627065225224' 2026-02-19T13:07:57.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:07:57.357 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:07:57.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=648540061698 2026-02-19T13:07:57.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 648540061698 2026-02-19T13:07:57.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-627065225224 1-648540061698' 2026-02-19T13:07:57.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:57.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-627065225224 2026-02-19T13:07:57.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:57.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:07:57.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-627065225224 2026-02-19T13:07:57.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:57.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=627065225224 2026-02-19T13:07:57.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 627065225224' 2026-02-19T13:07:57.437 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 627065225224 2026-02-19T13:07:57.437 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:57.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 627065225222 -lt 627065225224 2026-02-19T13:07:57.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:58.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:07:58.611 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:58.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 627065225222 -lt 627065225224 2026-02-19T13:07:58.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:07:59.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:07:59.770 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:07:59.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 627065225225 -lt 627065225224 2026-02-19T13:07:59.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:07:59.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-648540061698 2026-02-19T13:07:59.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:07:59.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:07:59.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-648540061698 2026-02-19T13:07:59.939 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:07:59.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=648540061698 2026-02-19T13:07:59.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 648540061698' 2026-02-19T13:07:59.940 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 648540061698 2026-02-19T13:07:59.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:00.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 648540061700 -lt 648540061698 2026-02-19T13:08:00.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:08:00.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:08:00.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:08:00.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:08:00.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:08:00.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:08:00.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:08:00.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:08:00.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:08:00.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:08:00.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:08:00.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:08:00.452 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:08:00.452 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:08:00.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:08:00.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:08:00.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:08:00.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:08:00.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:08:00.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ17 2026-02-19T13:08:00.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 17 % 2 2026-02-19T13:08:00.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=1 2026-02-19T13:08:00.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:08:00.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6575: TEST_dual_store_replicated_cluster: local payload=ROBJ17 2026-02-19T13:08:00.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6576: TEST_dual_store_replicated_cluster: echo ROBJ17 2026-02-19T13:08:00.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6577: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:08:00.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:08:00.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:08:00.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:08:00.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:08:00.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:08:00.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:08:00.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:08:00.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:08:00.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:08:00.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:08:00.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:08:00.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:08:00.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:08:00.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:08:00.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:08:00.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:08:00.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:08:00.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:08:00.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:08:00.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:08:00.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:08:00.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:08:00.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:08:01.143 INFO:tasks.workunit.client.0.trial144.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-19T13:08:01.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:08:01.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:08:01.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:08:01.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:08:01.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:08:01.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:08:01.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:08:01.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:08:01.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:08:01.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:08:01.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:08:01.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:08:01.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:08:01.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:08:01.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:08:01.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:08:01.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:08:01.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:08:01.240 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:08:01.240 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:01.240 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:01.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:01.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:01.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:08:01.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:08:01.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:08:01.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:08:01.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:08:01.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:08:01.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:08:01.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:08:01.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:08:01.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:08:01.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:08:01.243 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:08:01.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:08:01.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:08:01.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:08:01.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:08:01.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:08:01.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:08:01.264 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:01.262+0000 7f88387f7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:01.266 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:01.262+0000 7f88387f7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:01.266 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:01.266+0000 7f88387f7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:01.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:08:01.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:08:01.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:08:01.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:08:01.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:08:01.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:01.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:08:01.402 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:08:01.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:01.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:08:01.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:08:01.664 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:01.662+0000 7f88387f7a40 -1 Falling back to public interface 2026-02-19T13:08:02.257 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:02.254+0000 7f88387f7a40 -1 osd.0 152 log_to_monitors true 2026-02-19T13:08:02.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:08:02.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:02.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:08:02.571 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:08:02.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:02.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:08:02.726 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 156 up_thru 156 down_at 153 last_clean_interval [146,152) [v2:127.0.0.1:6802/2652940016,v1:127.0.0.1:6803/2652940016] [v2:127.0.0.1:6804/2652940016,v1:127.0.0.1:6805/2652940016] exists,up fe651040-8a56-4e0e-9bdd-37b3fbeea15d 2026-02-19T13:08:02.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:08:02.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:08:02.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:08:02.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:08:02.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:08:02.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:08:02.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:08:02.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:08:02.727 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:08:02.727 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:08:02.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:08:02.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:08:02.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:08:02.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:08:02.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:08:02.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:08:02.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:08:02.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:08:02.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:08:03.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:08:03.010 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:08:03.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:08:03.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:03.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:08:03.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=670014898178 2026-02-19T13:08:03.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 670014898178 2026-02-19T13:08:03.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-670014898178' 2026-02-19T13:08:03.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:03.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:08:03.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=648540061705 2026-02-19T13:08:03.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 648540061705 2026-02-19T13:08:03.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-670014898178 1-648540061705' 2026-02-19T13:08:03.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:03.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-670014898178 2026-02-19T13:08:03.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:03.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:08:03.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-670014898178 2026-02-19T13:08:03.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:03.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=670014898178 2026-02-19T13:08:03.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 670014898178' 2026-02-19T13:08:03.161 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 670014898178 2026-02-19T13:08:03.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:03.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 670014898178 -lt 670014898178 2026-02-19T13:08:03.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:03.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-648540061705 2026-02-19T13:08:03.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:03.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:08:03.328 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-648540061705 2026-02-19T13:08:03.328 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:03.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=648540061705 2026-02-19T13:08:03.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 648540061705' 2026-02-19T13:08:03.330 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 648540061705 2026-02-19T13:08:03.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:03.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 648540061704 -lt 648540061705 2026-02-19T13:08:03.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:08:04.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:08:04.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:04.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 648540061704 -lt 648540061705 2026-02-19T13:08:04.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:08:05.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:08:05.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:05.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 648540061707 -lt 648540061705 2026-02-19T13:08:05.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:08:05.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:08:05.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:08:06.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:08:06.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:08:06.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:08:06.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:08:06.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:08:06.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:08:06.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:08:06.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:08:06.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:08:06.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:08:06.185 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:08:06.185 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:08:06.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:08:06.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:08:06.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:08:06.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6578: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:08:06.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:08:06.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:08:06.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:08:06.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:08:06.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:08:06.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:08:06.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:08:06.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:08:06.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:08:06.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:08:06.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:08:06.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:08:06.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:08:06.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:08:06.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:08:06.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:08:06.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:08:06.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:08:06.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:08:06.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:08:06.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:08:06.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:08:06.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-19T13:08:06.868 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-19T13:08:06.868 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-19T13:08:06.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:08:06.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:08:06.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:08:06.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:08:06.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:08:06.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:08:06.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:08:06.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:08:06.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:08:06.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:08:06.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:08:06.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:08:06.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:08:06.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:08:06.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:08:06.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:08:06.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:08:06.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:08:06.971 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:08:06.971 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:06.971 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:06.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:06.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:06.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:08:06.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:08:06.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:08:06.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:08:06.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:08:06.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:08:06.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:08:06.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:08:06.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:08:06.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:08:06.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:08:06.974 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:08:06.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:08:06.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:08:06.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:08:06.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:08:06.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:08:06.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:08:06.993 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:06.990+0000 7fca2a99ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:06.994 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:06.990+0000 7fca2a99ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:06.995 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:06.994+0000 7fca2a99ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:07.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:08:07.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:08:07.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:08:07.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:08:07.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:08:07.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:07.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:08:07.132 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:08:07.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:07.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:08:07.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:08:07.336 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:07.334+0000 7fca2a99ca40 -1 Falling back to public interface 2026-02-19T13:08:08.038 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:08.034+0000 7fca2a99ca40 -1 osd.1 157 log_to_monitors true 2026-02-19T13:08:08.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:08:08.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:08.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:08:08.294 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:08:08.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:08.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:08:08.448 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 161 up_thru 161 down_at 158 last_clean_interval [151,157) [v2:127.0.0.1:6810/613469337,v1:127.0.0.1:6811/613469337] [v2:127.0.0.1:6812/613469337,v1:127.0.0.1:6813/613469337] exists,up c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:08:08.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:08:08.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:08:08.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:08:08.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:08:08.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:08:08.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:08:08.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:08:08.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:08:08.450 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:08:08.450 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:08:08.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:08:08.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:08:08.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:08:08.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:08:08.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:08:08.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:08:08.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:08:08.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:08:08.566 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:08:08.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:08:08.729 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:08:08.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:08:08.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:08.729 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:08:08.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=670014898185 2026-02-19T13:08:08.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 670014898185 2026-02-19T13:08:08.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-670014898185' 2026-02-19T13:08:08.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:08.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:08:08.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=691489734658 2026-02-19T13:08:08.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 691489734658 2026-02-19T13:08:08.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-670014898185 1-691489734658' 2026-02-19T13:08:08.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:08.876 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-670014898185 2026-02-19T13:08:08.876 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:08.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:08:08.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-670014898185 2026-02-19T13:08:08.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:08.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=670014898185 2026-02-19T13:08:08.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 670014898185' 2026-02-19T13:08:08.879 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 670014898185 2026-02-19T13:08:08.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:09.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 670014898182 -lt 670014898185 2026-02-19T13:08:09.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:08:10.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:08:10.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:10.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 670014898185 -lt 670014898185 2026-02-19T13:08:10.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:10.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-691489734658 2026-02-19T13:08:10.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:10.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:08:10.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-691489734658 2026-02-19T13:08:10.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:10.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=691489734658 2026-02-19T13:08:10.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 691489734658' 2026-02-19T13:08:10.204 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 691489734658 2026-02-19T13:08:10.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:10.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 691489734658 -lt 691489734658 2026-02-19T13:08:10.366 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:08:10.366 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:08:10.366 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:08:10.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:08:10.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:08:10.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:08:10.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:08:10.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:08:10.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:08:10.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:08:10.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:08:10.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:08:10.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:08:10.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:08:10.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:08:10.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:08:10.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:08:10.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:08:10.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:08:10.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ18 2026-02-19T13:08:10.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 18 % 2 2026-02-19T13:08:10.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=0 2026-02-19T13:08:10.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:08:10.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6583: TEST_dual_store_replicated_cluster: local payload=ROBJ18 2026-02-19T13:08:10.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6584: TEST_dual_store_replicated_cluster: echo ROBJ18 2026-02-19T13:08:10.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6585: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:08:10.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:08:10.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:08:10.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:08:10.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:08:10.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:08:10.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:08:10.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:08:10.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:08:10.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:08:10.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:08:10.920 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:08:10.920 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:08:10.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:08:10.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:08:10.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:08:11.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:08:11.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:08:11.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:08:11.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:08:11.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:08:11.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:08:11.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:08:11.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:08:11.399 INFO:tasks.workunit.client.0.trial144.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-19T13:08:11.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:08:11.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:08:11.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:08:11.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:08:11.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:08:11.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:08:11.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:08:11.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:08:11.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:08:11.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:08:11.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:08:11.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:08:11.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:08:11.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:08:11.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:08:11.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:08:11.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:08:11.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:08:11.492 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:08:11.492 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:11.492 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:11.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:11.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:11.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:08:11.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:08:11.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:08:11.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:08:11.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:08:11.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:08:11.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:08:11.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:08:11.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:08:11.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:08:11.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:08:11.494 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:08:11.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:08:11.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:08:11.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:08:11.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:08:11.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:08:11.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:08:11.514 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:11.510+0000 7fca2d466a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:11.515 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:11.514+0000 7fca2d466a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:11.516 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:11.514+0000 7fca2d466a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:11.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:08:11.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:08:11.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:08:11.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:08:11.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:08:11.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:11.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:08:11.653 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:08:11.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:11.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:08:11.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:08:11.868 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:11.866+0000 7fca2d466a40 -1 Falling back to public interface 2026-02-19T13:08:12.506 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:12.502+0000 7fca2d466a40 -1 osd.0 162 log_to_monitors true 2026-02-19T13:08:12.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:08:12.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:12.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:08:12.814 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:08:12.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:12.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:08:12.968 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 166 up_thru 166 down_at 163 last_clean_interval [156,162) [v2:127.0.0.1:6802/3323448256,v1:127.0.0.1:6803/3323448256] [v2:127.0.0.1:6804/3323448256,v1:127.0.0.1:6805/3323448256] exists,up fe651040-8a56-4e0e-9bdd-37b3fbeea15d 2026-02-19T13:08:12.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:08:12.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:08:12.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:08:12.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:08:12.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:08:12.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:08:12.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:08:12.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:08:12.970 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:08:12.970 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:08:12.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:08:12.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:08:12.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:08:13.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:08:13.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:08:13.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:08:13.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:08:13.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:08:13.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:08:13.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:08:13.242 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:08:13.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:08:13.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:13.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:08:13.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=712964571138 2026-02-19T13:08:13.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 712964571138 2026-02-19T13:08:13.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571138' 2026-02-19T13:08:13.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:13.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:08:13.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=691489734664 2026-02-19T13:08:13.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 691489734664 2026-02-19T13:08:13.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571138 1-691489734664' 2026-02-19T13:08:13.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:13.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-712964571138 2026-02-19T13:08:13.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:13.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:08:13.388 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-712964571138 2026-02-19T13:08:13.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:13.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=712964571138 2026-02-19T13:08:13.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 712964571138' 2026-02-19T13:08:13.390 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 712964571138 2026-02-19T13:08:13.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:13.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 712964571138 2026-02-19T13:08:13.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:08:14.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:08:14.557 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:14.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 712964571138 2026-02-19T13:08:14.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:08:15.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:08:15.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:15.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571140 -lt 712964571138 2026-02-19T13:08:15.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:15.882 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-691489734664 2026-02-19T13:08:15.882 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:15.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:08:15.884 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-691489734664 2026-02-19T13:08:15.884 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:15.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=691489734664 2026-02-19T13:08:15.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 691489734664' 2026-02-19T13:08:15.886 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 691489734664 2026-02-19T13:08:15.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:16.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 691489734665 -lt 691489734664 2026-02-19T13:08:16.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:08:16.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:08:16.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:08:16.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:08:16.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:08:16.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:08:16.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:08:16.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:08:16.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:08:16.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:08:16.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:08:16.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:08:16.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:08:16.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:08:16.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6586: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:08:16.608 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:08:16.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:08:16.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:08:16.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:08:16.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:08:16.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:08:16.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:08:16.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:08:16.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:08:16.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:08:16.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:08:16.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-19T13:08:17.108 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-19T13:08:17.108 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-19T13:08:17.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:08:17.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:08:17.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:08:17.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:08:17.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:08:17.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:08:17.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:08:17.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:08:17.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:08:17.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:08:17.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:08:17.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:08:17.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:08:17.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:08:17.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:08:17.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:08:17.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:08:17.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:08:17.198 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:08:17.199 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:17.199 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:17.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:17.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:17.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:08:17.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:08:17.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:08:17.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:08:17.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:08:17.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:08:17.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:08:17.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:08:17.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:08:17.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:08:17.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:08:17.202 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:08:17.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:08:17.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:08:17.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:08:17.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:08:17.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:08:17.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:08:17.221 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:17.218+0000 7fd2b8a83a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:17.222 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:17.218+0000 7fd2b8a83a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:17.223 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:17.222+0000 7fd2b8a83a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:17.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:08:17.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:08:17.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:08:17.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:08:17.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:08:17.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:17.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:08:17.366 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:08:17.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:17.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:08:17.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:08:17.644 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:17.642+0000 7fd2b8a83a40 -1 Falling back to public interface 2026-02-19T13:08:18.292 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:18.290+0000 7fd2b8a83a40 -1 osd.1 167 log_to_monitors true 2026-02-19T13:08:18.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:08:18.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:18.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:08:18.525 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:08:18.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:18.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:08:18.684 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 171 up_thru 171 down_at 168 last_clean_interval [161,167) [v2:127.0.0.1:6810/2003812566,v1:127.0.0.1:6811/2003812566] [v2:127.0.0.1:6812/2003812566,v1:127.0.0.1:6813/2003812566] exists,up c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:08:18.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:08:18.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:08:18.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:08:18.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:08:18.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:08:18.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:08:18.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:08:18.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:08:18.686 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:08:18.686 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:08:18.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:08:18.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:08:18.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:08:18.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:08:18.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:08:18.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:08:18.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:08:18.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:08:18.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:08:18.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:08:18.966 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:08:18.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:08:18.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:18.966 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:08:19.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=712964571145 2026-02-19T13:08:19.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 712964571145 2026-02-19T13:08:19.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571145' 2026-02-19T13:08:19.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:19.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:08:19.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=734439407618 2026-02-19T13:08:19.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 734439407618 2026-02-19T13:08:19.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571145 1-734439407618' 2026-02-19T13:08:19.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:19.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-712964571145 2026-02-19T13:08:19.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:19.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:08:19.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-712964571145 2026-02-19T13:08:19.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:19.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=712964571145 2026-02-19T13:08:19.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 712964571145' 2026-02-19T13:08:19.118 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 712964571145 2026-02-19T13:08:19.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:19.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571145 -lt 712964571145 2026-02-19T13:08:19.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:19.287 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-734439407618 2026-02-19T13:08:19.287 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:19.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:08:19.289 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-734439407618 2026-02-19T13:08:19.289 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:19.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=734439407618 2026-02-19T13:08:19.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 734439407618' 2026-02-19T13:08:19.291 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 734439407618 2026-02-19T13:08:19.292 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:19.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 734439407618 2026-02-19T13:08:19.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:08:20.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:08:20.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:20.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 734439407618 2026-02-19T13:08:20.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:08:21.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:08:21.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:21.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 734439407620 -lt 734439407618 2026-02-19T13:08:21.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:08:21.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:08:21.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:08:21.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:08:21.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:08:21.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:08:21.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:08:21.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:08:21.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:08:21.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:08:21.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:08:22.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:08:22.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:08:22.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:08:22.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:08:22.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:08:22.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:08:22.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:08:22.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6588: TEST_dual_store_replicated_cluster: objectstore_tool td/osd-scrub-repair 1 ROBJ18 corrupt-info 2026-02-19T13:08:22.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:08:22.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:08:22.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:08:22.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:08:22.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ18 corrupt-info 2026-02-19T13:08:22.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:08:22.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:08:22.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:08:22.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:08:22.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:08:22.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:08:22.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:08:22.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:08:22.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:08:22.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:08:22.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:08:22.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ18 corrupt-info 2026-02-19T13:08:22.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:08:22.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:08:22.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:08:22.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:08:22.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:08:22.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ18 corrupt-info 2026-02-19T13:08:22.814 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-19T13:08:22.814 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-19T13:08:22.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:08:22.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:08:22.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:08:22.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:08:22.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:08:22.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:08:22.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:08:22.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:08:22.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:08:22.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:08:22.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:08:22.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:08:22.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:08:22.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:08:22.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:08:22.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:08:22.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:08:22.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:08:22.904 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:08:22.904 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:22.904 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:22.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:22.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:22.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:08:22.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:08:22.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:08:22.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:08:22.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:08:22.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:08:22.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:08:22.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:08:22.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0' 2026-02-19T13:08:22.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:08:22.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:08:22.906 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:08:22.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_op_queue=wpq --osd_scrub_auto_repair=0 2026-02-19T13:08:22.907 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:08:22.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:08:22.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:08:22.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:08:22.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:08:22.928 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:22.926+0000 7fe2b106fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:22.929 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:22.926+0000 7fe2b106fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:22.930 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:22.926+0000 7fe2b106fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:23.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:08:23.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:08:23.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:08:23.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:08:23.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:08:23.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:23.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:08:23.069 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:08:23.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:23.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:08:23.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:08:23.289 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:23.286+0000 7fe2b106fa40 -1 Falling back to public interface 2026-02-19T13:08:23.952 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:23.950+0000 7fe2b106fa40 -1 osd.1 172 log_to_monitors true 2026-02-19T13:08:24.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:08:24.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:24.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:08:24.229 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:08:24.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:24.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:08:24.384 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 176 up_thru 176 down_at 173 last_clean_interval [171,172) [v2:127.0.0.1:6810/1596219048,v1:127.0.0.1:6811/1596219048] [v2:127.0.0.1:6812/1596219048,v1:127.0.0.1:6813/1596219048] exists,up c8f44563-f119-46be-a0f5-9bd991c666ec 2026-02-19T13:08:24.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:08:24.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:08:24.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:08:24.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:08:24.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:08:24.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:08:24.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:08:24.385 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:08:24.386 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:08:24.386 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:08:24.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:08:24.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:08:24.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:08:24.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:08:24.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:08:24.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:08:24.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:08:24.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:08:24.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:08:24.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:08:24.670 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:08:24.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:08:24.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:24.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:08:24.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=712964571152 2026-02-19T13:08:24.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 712964571152 2026-02-19T13:08:24.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571152' 2026-02-19T13:08:24.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:24.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:08:24.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=755914244098 2026-02-19T13:08:24.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 755914244098 2026-02-19T13:08:24.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571152 1-755914244098' 2026-02-19T13:08:24.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:24.816 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-712964571152 2026-02-19T13:08:24.816 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:24.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:08:24.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-712964571152 2026-02-19T13:08:24.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:24.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=712964571152 2026-02-19T13:08:24.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 712964571152' 2026-02-19T13:08:24.820 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 712964571152 2026-02-19T13:08:24.820 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:24.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571149 -lt 712964571152 2026-02-19T13:08:24.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:08:25.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:08:25.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:26.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571152 -lt 712964571152 2026-02-19T13:08:26.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:26.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-755914244098 2026-02-19T13:08:26.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:26.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:08:26.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-755914244098 2026-02-19T13:08:26.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:26.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=755914244098 2026-02-19T13:08:26.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 755914244098' 2026-02-19T13:08:26.139 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 755914244098 2026-02-19T13:08:26.139 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:26.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 755914244099 -lt 755914244098 2026-02-19T13:08:26.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:08:26.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:08:26.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:08:26.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:08:26.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:08:26.487 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:08:26.487 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:08:26.487 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:08:26.487 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:08:26.488 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:08:26.488 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:08:26.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:08:26.641 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:08:26.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:08:26.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:08:26.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:08:26.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:08:26.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:08:26.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6480: TEST_dual_store_replicated_cluster: for i in $(seq 1 $total_objs) 2026-02-19T13:08:26.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6481: TEST_dual_store_replicated_cluster: objname=ROBJ19 2026-02-19T13:08:26.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: expr 19 % 2 2026-02-19T13:08:26.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6484: TEST_dual_store_replicated_cluster: local osd=1 2026-02-19T13:08:26.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6486: TEST_dual_store_replicated_cluster: case $i in 2026-02-19T13:08:26.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6597: TEST_dual_store_replicated_cluster: get_pg csr_pool ROBJ0 2026-02-19T13:08:26.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=csr_pool 2026-02-19T13:08:26.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=ROBJ0 2026-02-19T13:08:26.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map csr_pool ROBJ0 2026-02-19T13:08:26.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:08:26.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6597: TEST_dual_store_replicated_cluster: local pg=3.0 2026-02-19T13:08:26.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6599: TEST_dual_store_replicated_cluster: ceph tell 'osd.*' injectargs -- --osd-max-object-size=1048576 2026-02-19T13:08:27.059 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: {} 2026-02-19T13:08:27.060 INFO:tasks.workunit.client.0.trial144.stderr:osd.0: osd_max_object_size = '' 2026-02-19T13:08:27.068 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: {} 2026-02-19T13:08:27.068 INFO:tasks.workunit.client.0.trial144.stderr:osd.1: osd_max_object_size = '' 2026-02-19T13:08:27.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6601: TEST_dual_store_replicated_cluster: inject_eio rep data csr_pool ROBJ11 td/osd-scrub-repair 0 2026-02-19T13:08:27.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=rep 2026-02-19T13:08:27.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:08:27.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:08:27.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:08:27.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=csr_pool 2026-02-19T13:08:27.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:08:27.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=ROBJ11 2026-02-19T13:08:27.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:08:27.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/osd-scrub-repair 2026-02-19T13:08:27.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:08:27.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-19T13:08:27.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:08:27.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds csr_pool ROBJ11 2026-02-19T13:08:27.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=csr_pool 2026-02-19T13:08:27.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=ROBJ11 2026-02-19T13:08:27.079 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map csr_pool ROBJ11 2026-02-19T13:08:27.080 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:08:27.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-19T13:08:27.243 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:08:27.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-19T13:08:27.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('1' '0') 2026-02-19T13:08:27.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:08:27.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-19T13:08:27.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' rep '!=' ec ']' 2026-02-19T13:08:27.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2497: inject_eio: shard_id= 2026-02-19T13:08:27.245 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/osd-scrub-repair/1/type 2026-02-19T13:08:27.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:08:27.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T13:08:27.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:08:27.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T13:08:27.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:08:27.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:08:27.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:08:27.246 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T13:08:27.246 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:08:27.247 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:08:27.247 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:08:27.247 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:27.248 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:27.248 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T13:08:27.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:08:27.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:08:27.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:08:27.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:08:27.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-19T13:08:27.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:08:27.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:08:27.310 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:08:27.310 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:27.310 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:27.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T13:08:27.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:08:27.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.35818/ceph-osd.1.asok injectdataerr csr_pool ROBJ11 2026-02-19T13:08:27.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6602: TEST_dual_store_replicated_cluster: inject_eio rep mdata csr_pool ROBJ12 td/osd-scrub-repair 1 2026-02-19T13:08:27.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=rep 2026-02-19T13:08:27.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:08:27.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=mdata 2026-02-19T13:08:27.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:08:27.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=csr_pool 2026-02-19T13:08:27.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:08:27.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=ROBJ12 2026-02-19T13:08:27.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:08:27.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/osd-scrub-repair 2026-02-19T13:08:27.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:08:27.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-19T13:08:27.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:08:27.371 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds csr_pool ROBJ12 2026-02-19T13:08:27.371 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=csr_pool 2026-02-19T13:08:27.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=ROBJ12 2026-02-19T13:08:27.372 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map csr_pool ROBJ12 2026-02-19T13:08:27.372 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:08:27.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-19T13:08:27.533 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:08:27.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-19T13:08:27.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('1' '0') 2026-02-19T13:08:27.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:08:27.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=0 2026-02-19T13:08:27.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' rep '!=' ec ']' 2026-02-19T13:08:27.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2497: inject_eio: shard_id= 2026-02-19T13:08:27.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/osd-scrub-repair/0/type 2026-02-19T13:08:27.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:08:27.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-19T13:08:27.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:08:27.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=0 2026-02-19T13:08:27.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:08:27.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:08:27.536 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:08:27.537 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.0 2026-02-19T13:08:27.537 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:08:27.537 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:08:27.537 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:08:27.537 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:27.537 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:27.537 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.0.asok 2026-02-19T13:08:27.537 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:08:27.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:08:27.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:08:27.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:08:27.598 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.0 2026-02-19T13:08:27.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:08:27.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:08:27.599 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:08:27.599 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:27.599 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:27.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.0.asok 2026-02-19T13:08:27.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:08:27.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.35818/ceph-osd.0.asok injectmdataerr csr_pool ROBJ12 2026-02-19T13:08:27.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6603: TEST_dual_store_replicated_cluster: inject_eio rep data csr_pool ROBJ13 td/osd-scrub-repair 0 2026-02-19T13:08:27.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=rep 2026-02-19T13:08:27.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:08:27.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:08:27.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:08:27.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=csr_pool 2026-02-19T13:08:27.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:08:27.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=ROBJ13 2026-02-19T13:08:27.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:08:27.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/osd-scrub-repair 2026-02-19T13:08:27.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:08:27.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-19T13:08:27.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:08:27.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds csr_pool ROBJ13 2026-02-19T13:08:27.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=csr_pool 2026-02-19T13:08:27.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=ROBJ13 2026-02-19T13:08:27.660 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map csr_pool ROBJ13 2026-02-19T13:08:27.660 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:08:27.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-19T13:08:27.822 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:08:27.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-19T13:08:27.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('1' '0') 2026-02-19T13:08:27.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:08:27.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-19T13:08:27.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' rep '!=' ec ']' 2026-02-19T13:08:27.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2497: inject_eio: shard_id= 2026-02-19T13:08:27.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/osd-scrub-repair/1/type 2026-02-19T13:08:27.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:08:27.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T13:08:27.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:08:27.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T13:08:27.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:08:27.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:08:27.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:08:27.827 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T13:08:27.827 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:08:27.827 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:08:27.828 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:08:27.828 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:27.828 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:27.828 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T13:08:27.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:08:27.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:08:27.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:08:27.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:08:27.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-19T13:08:27.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:08:27.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:08:27.890 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:08:27.890 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:27.890 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:27.890 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.1.asok 2026-02-19T13:08:27.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:08:27.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.35818/ceph-osd.1.asok injectdataerr csr_pool ROBJ13 2026-02-19T13:08:27.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6606: TEST_dual_store_replicated_cluster: pg_scrub 3.0 2026-02-19T13:08:27.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=3.0 2026-02-19T13:08:27.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 3.0 2026-02-19T13:08:27.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=3.0 2026-02-19T13:08:27.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:08:27.950 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:08:27.950 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:08:27.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:08:27.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:08:27.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:08:28.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:08:28.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:08:28.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:08:28.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:08:28.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:08:28.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:08:28.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:08:28.418 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:08:28.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:08:28.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:28.418 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:08:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=712964571156 2026-02-19T13:08:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 712964571156 2026-02-19T13:08:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571156' 2026-02-19T13:08:28.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:28.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:08:28.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=755914244103 2026-02-19T13:08:28.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 755914244103 2026-02-19T13:08:28.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571156 1-755914244103' 2026-02-19T13:08:28.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:28.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-712964571156 2026-02-19T13:08:28.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:28.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:08:28.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-712964571156 2026-02-19T13:08:28.566 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:28.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=712964571156 2026-02-19T13:08:28.567 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 712964571156 2026-02-19T13:08:28.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 712964571156' 2026-02-19T13:08:28.568 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:28.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571154 -lt 712964571156 2026-02-19T13:08:28.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:08:29.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:08:29.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:29.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571157 -lt 712964571156 2026-02-19T13:08:29.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:29.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-755914244103 2026-02-19T13:08:29.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:29.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:08:29.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-755914244103 2026-02-19T13:08:29.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:29.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=755914244103 2026-02-19T13:08:29.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 755914244103' 2026-02-19T13:08:29.903 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 755914244103 2026-02-19T13:08:29.903 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:30.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 755914244104 -lt 755914244103 2026-02-19T13:08:30.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:08:30.066 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 3.0 loop 0 2026-02-19T13:08:30.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-19T13:08:30.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 3.0 2026-02-19T13:08:30.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=3.0 2026-02-19T13:08:30.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:08:30.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 3.0 query 2026-02-19T13:08:30.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:08:30.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-19T13:08:30.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:08:30.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:08:30.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:08:30.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 3.0 2026-02-19T13:08:30.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:30.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:08:30.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:30.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:08:30.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-19T13:06:33.452816+0000 2026-02-19T13:08:30.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 3.0 2026-02-19T13:08:30.446 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0 on osd.1 to scrub 2026-02-19T13:08:30.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 3.0 2026-02-19T13:06:33.452816+0000 2026-02-19T13:08:30.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:08:30.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:06:33.452816+0000 2026-02-19T13:08:30.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:08:30.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:08:30.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:08:30.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:08:30.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:30.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:08:30.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:30.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:08:30.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:06:33.452816+0000 '>' 2026-02-19T13:06:33.452816+0000 2026-02-19T13:08:30.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:08:31.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:08:31.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:08:31.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:08:31.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:31.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:08:31.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:31.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:08:31.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:08:31.019540+0000 '>' 2026-02-19T13:06:33.452816+0000 2026-02-19T13:08:31.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:08:31.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6607: TEST_dual_store_replicated_cluster: (( extr_dbg >= 3 )) 2026-02-19T13:08:31.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6608: TEST_dual_store_replicated_cluster: pg_scrub 3.0 2026-02-19T13:08:31.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=3.0 2026-02-19T13:08:31.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 3.0 2026-02-19T13:08:31.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=3.0 2026-02-19T13:08:31.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:08:31.782 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:08:31.782 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:08:31.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:08:31.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:08:31.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:08:32.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:08:32.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:08:32.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:08:32.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:08:32.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:08:32.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:08:32.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:08:32.238 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:08:32.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:08:32.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:32.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:08:32.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=712964571161 2026-02-19T13:08:32.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 712964571161 2026-02-19T13:08:32.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571161' 2026-02-19T13:08:32.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:32.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:08:32.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=755914244108 2026-02-19T13:08:32.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 755914244108 2026-02-19T13:08:32.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571161 1-755914244108' 2026-02-19T13:08:32.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:32.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-712964571161 2026-02-19T13:08:32.383 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:32.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:08:32.385 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-712964571161 2026-02-19T13:08:32.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:32.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=712964571161 2026-02-19T13:08:32.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 712964571161' 2026-02-19T13:08:32.387 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 712964571161 2026-02-19T13:08:32.388 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:32.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571159 -lt 712964571161 2026-02-19T13:08:32.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:08:33.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:08:33.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:33.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571162 -lt 712964571161 2026-02-19T13:08:33.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:33.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-755914244108 2026-02-19T13:08:33.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:33.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:08:33.711 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-755914244108 2026-02-19T13:08:33.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:33.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=755914244108 2026-02-19T13:08:33.713 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 755914244108 2026-02-19T13:08:33.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 755914244108' 2026-02-19T13:08:33.714 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:33.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 755914244109 -lt 755914244108 2026-02-19T13:08:33.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:08:33.882 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 3.0 loop 0 2026-02-19T13:08:33.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-19T13:08:33.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 3.0 2026-02-19T13:08:33.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=3.0 2026-02-19T13:08:33.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:08:33.883 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 3.0 query 2026-02-19T13:08:33.883 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:08:33.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean+inconsistent 2026-02-19T13:08:33.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean+inconsistent == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:08:33.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:08:33.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:08:33.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 3.0 2026-02-19T13:08:33.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:33.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:08:33.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:33.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:08:34.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-19T13:08:31.019540+0000 2026-02-19T13:08:34.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 3.0 2026-02-19T13:08:34.255 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0 on osd.1 to scrub 2026-02-19T13:08:34.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 3.0 2026-02-19T13:08:31.019540+0000 2026-02-19T13:08:34.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:08:34.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:08:31.019540+0000 2026-02-19T13:08:34.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:08:34.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:08:34.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:08:34.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:08:34.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:34.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:08:34.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:34.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:08:34.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:08:31.019540+0000 '>' 2026-02-19T13:08:31.019540+0000 2026-02-19T13:08:34.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:08:35.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:08:35.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:08:35.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:08:35.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:35.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:08:35.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:35.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:08:35.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:08:34.997108+0000 '>' 2026-02-19T13:08:31.019540+0000 2026-02-19T13:08:35.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:08:35.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6609: TEST_dual_store_replicated_cluster: (( extr_dbg >= 3 )) 2026-02-19T13:08:35.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6610: TEST_dual_store_replicated_cluster: rados list-inconsistent-obj 3.0 2026-02-19T13:08:35.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6610: TEST_dual_store_replicated_cluster: jq .inconsistents 2026-02-19T13:08:35.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6610: TEST_dual_store_replicated_cluster: jq 'def walk(f): 2026-02-19T13:08:35.584 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:08:35.584 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:08:35.584 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:08:35.584 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:08:35.584 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:08:35.584 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:08:35.584 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:08:35.584 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:08:35.584 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:08:35.584 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:08:35.585 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:08:35.585 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-19T13:08:35.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6610: TEST_dual_store_replicated_cluster: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:08:35.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6611: TEST_dual_store_replicated_cluster: (( extr_dbg >= 3 )) 2026-02-19T13:08:35.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6614: TEST_dual_store_replicated_cluster: pg_deep_scrub 3.0 2026-02-19T13:08:35.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_deep_scrub: local pgid=3.0 2026-02-19T13:08:35.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_deep_scrub: wait_for_pg_clean 3.0 2026-02-19T13:08:35.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=3.0 2026-02-19T13:08:35.613 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:08:35.613 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:08:35.613 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:08:35.613 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:08:35.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:08:35.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:08:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:08:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:08:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:08:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:08:35.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:08:35.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:08:36.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:08:36.082 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:08:36.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:08:36.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:36.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:08:36.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=712964571166 2026-02-19T13:08:36.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 712964571166 2026-02-19T13:08:36.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571166' 2026-02-19T13:08:36.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:36.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:08:36.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=755914244113 2026-02-19T13:08:36.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 755914244113 2026-02-19T13:08:36.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571166 1-755914244113' 2026-02-19T13:08:36.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:36.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-712964571166 2026-02-19T13:08:36.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:36.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:08:36.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-712964571166 2026-02-19T13:08:36.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:36.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=712964571166 2026-02-19T13:08:36.231 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 712964571166 2026-02-19T13:08:36.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 712964571166' 2026-02-19T13:08:36.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:36.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571164 -lt 712964571166 2026-02-19T13:08:36.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:08:37.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:08:37.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:37.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571167 -lt 712964571166 2026-02-19T13:08:37.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:37.555 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-755914244113 2026-02-19T13:08:37.555 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:37.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:08:37.557 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-755914244113 2026-02-19T13:08:37.557 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:37.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=755914244113 2026-02-19T13:08:37.559 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 755914244113 2026-02-19T13:08:37.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 755914244113' 2026-02-19T13:08:37.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:37.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 755914244114 -lt 755914244113 2026-02-19T13:08:37.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:08:37.722 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 3.0 loop 0 2026-02-19T13:08:37.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-19T13:08:37.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 3.0 2026-02-19T13:08:37.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=3.0 2026-02-19T13:08:37.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:08:37.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 3.0 query 2026-02-19T13:08:37.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:08:37.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean+inconsistent 2026-02-19T13:08:37.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean+inconsistent == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:08:37.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:08:37.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:08:37.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:08:37.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:37.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:08:37.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:37.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:08:37.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: local last_scrub=2026-02-19T13:06:33.452816+0000 2026-02-19T13:08:37.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_deep_scrub: ceph pg deep-scrub 3.0 2026-02-19T13:08:38.105 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2026-02-19T13:08:38.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_deep_scrub: wait_for_scrub 3.0 2026-02-19T13:06:33.452816+0000 last_deep_scrub_stamp 2026-02-19T13:08:38.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:08:38.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:06:33.452816+0000 2026-02-19T13:08:38.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-19T13:08:38.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:08:38.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:08:38.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:08:38.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:38.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:08:38.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:38.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:08:38.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:06:33.452816+0000 '>' 2026-02-19T13:06:33.452816+0000 2026-02-19T13:08:38.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:08:39.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:08:39.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:08:39.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:08:39.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:39.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:08:39.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:39.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:08:39.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:08:39.017771+0000 '>' 2026-02-19T13:06:33.452816+0000 2026-02-19T13:08:39.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:08:39.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6615: TEST_dual_store_replicated_cluster: (( extr_dbg >= 3 )) 2026-02-19T13:08:39.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6616: TEST_dual_store_replicated_cluster: rados list-inconsistent-obj 3.0 2026-02-19T13:08:39.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6616: TEST_dual_store_replicated_cluster: jq .inconsistents 2026-02-19T13:08:39.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6616: TEST_dual_store_replicated_cluster: jq 'def walk(f): 2026-02-19T13:08:39.444 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:08:39.444 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:08:39.444 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:08:39.444 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:08:39.444 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:08:39.444 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:08:39.444 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:08:39.444 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:08:39.444 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:08:39.444 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:08:39.445 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:08:39.445 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-19T13:08:39.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6616: TEST_dual_store_replicated_cluster: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:08:39.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6617: TEST_dual_store_replicated_cluster: (( extr_dbg >= 3 )) 2026-02-19T13:08:39.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6620: TEST_dual_store_replicated_cluster: pg_scrub 3.0 2026-02-19T13:08:39.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=3.0 2026-02-19T13:08:39.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 3.0 2026-02-19T13:08:39.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=3.0 2026-02-19T13:08:39.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:08:39.476 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:08:39.476 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:08:39.477 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:08:39.477 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:08:39.477 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:08:39.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:08:39.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:08:39.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:08:39.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:08:39.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:08:39.791 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:08:39.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:08:39.951 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:08:39.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:08:39.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:39.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:08:40.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=712964571171 2026-02-19T13:08:40.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 712964571171 2026-02-19T13:08:40.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571171' 2026-02-19T13:08:40.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:40.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:08:40.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=755914244118 2026-02-19T13:08:40.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 755914244118 2026-02-19T13:08:40.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571171 1-755914244118' 2026-02-19T13:08:40.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:40.101 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-712964571171 2026-02-19T13:08:40.101 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:40.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:08:40.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-712964571171 2026-02-19T13:08:40.104 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:40.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=712964571171 2026-02-19T13:08:40.105 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 712964571171 2026-02-19T13:08:40.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 712964571171' 2026-02-19T13:08:40.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:40.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571169 -lt 712964571171 2026-02-19T13:08:40.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:08:41.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:08:41.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:41.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571172 -lt 712964571171 2026-02-19T13:08:41.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:41.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-755914244118 2026-02-19T13:08:41.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:41.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:08:41.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-755914244118 2026-02-19T13:08:41.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:41.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=755914244118 2026-02-19T13:08:41.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 755914244118' 2026-02-19T13:08:41.430 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 755914244118 2026-02-19T13:08:41.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:41.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 755914244119 -lt 755914244118 2026-02-19T13:08:41.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:08:41.603 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 3.0 loop 0 2026-02-19T13:08:41.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-19T13:08:41.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 3.0 2026-02-19T13:08:41.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=3.0 2026-02-19T13:08:41.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:08:41.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 3.0 query 2026-02-19T13:08:41.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:08:41.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean+inconsistent 2026-02-19T13:08:41.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean+inconsistent == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:08:41.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:08:41.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:08:41.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 3.0 2026-02-19T13:08:41.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:41.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:08:41.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:41.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:08:41.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-19T13:08:39.017771+0000 2026-02-19T13:08:41.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 3.0 2026-02-19T13:08:41.985 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0 on osd.1 to scrub 2026-02-19T13:08:41.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 3.0 2026-02-19T13:08:39.017771+0000 2026-02-19T13:08:41.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:08:41.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:08:39.017771+0000 2026-02-19T13:08:41.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:08:41.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:08:41.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:08:41.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:08:41.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:41.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:08:41.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:41.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:08:42.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:08:39.017771+0000 '>' 2026-02-19T13:08:39.017771+0000 2026-02-19T13:08:42.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:08:43.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:08:43.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:08:43.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:08:43.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:43.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:08:43.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:43.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:08:43.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:08:42.945912+0000 '>' 2026-02-19T13:08:39.017771+0000 2026-02-19T13:08:43.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:08:43.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6621: TEST_dual_store_replicated_cluster: (( extr_dbg >= 3 )) 2026-02-19T13:08:43.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6622: TEST_dual_store_replicated_cluster: rados list-inconsistent-obj 3.0 2026-02-19T13:08:43.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6622: TEST_dual_store_replicated_cluster: jq .inconsistents 2026-02-19T13:08:43.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6622: TEST_dual_store_replicated_cluster: jq 'def walk(f): 2026-02-19T13:08:43.322 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:08:43.322 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:08:43.322 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:08:43.322 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:08:43.322 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:08:43.322 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:08:43.322 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:08:43.322 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:08:43.323 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:08:43.323 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:08:43.323 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:08:43.323 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-19T13:08:43.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6622: TEST_dual_store_replicated_cluster: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:08:43.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6623: TEST_dual_store_replicated_cluster: (( extr_dbg >= 3 )) 2026-02-19T13:08:43.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6626: TEST_dual_store_replicated_cluster: diff -u td/osd-scrub-repair/dp_results.json td/osd-scrub-repair/sh2_results.json 2026-02-19T13:08:43.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6630: TEST_dual_store_replicated_cluster: inject_eio rep mdata csr_pool ROBJ13 td/osd-scrub-repair 1 2026-02-19T13:08:43.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=rep 2026-02-19T13:08:43.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:08:43.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=mdata 2026-02-19T13:08:43.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:08:43.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=csr_pool 2026-02-19T13:08:43.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:08:43.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=ROBJ13 2026-02-19T13:08:43.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:08:43.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/osd-scrub-repair 2026-02-19T13:08:43.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:08:43.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-19T13:08:43.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:08:43.357 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds csr_pool ROBJ13 2026-02-19T13:08:43.357 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=csr_pool 2026-02-19T13:08:43.357 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=ROBJ13 2026-02-19T13:08:43.358 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map csr_pool ROBJ13 2026-02-19T13:08:43.358 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:08:43.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-19T13:08:43.515 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:08:43.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-19T13:08:43.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('1' '0') 2026-02-19T13:08:43.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:08:43.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=0 2026-02-19T13:08:43.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' rep '!=' ec ']' 2026-02-19T13:08:43.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2497: inject_eio: shard_id= 2026-02-19T13:08:43.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/osd-scrub-repair/0/type 2026-02-19T13:08:43.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:08:43.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-19T13:08:43.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:08:43.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=0 2026-02-19T13:08:43.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:08:43.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:08:43.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:08:43.518 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.0 2026-02-19T13:08:43.518 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:08:43.519 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:08:43.519 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:08:43.519 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:43.519 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:43.519 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.0.asok 2026-02-19T13:08:43.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.35818/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:08:43.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:08:43.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:08:43.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:08:43.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.0 2026-02-19T13:08:43.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:08:43.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:08:43.583 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:08:43.583 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:43.584 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:43.584 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-osd.0.asok 2026-02-19T13:08:43.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:08:43.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.35818/ceph-osd.0.asok injectmdataerr csr_pool ROBJ13 2026-02-19T13:08:43.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6632: TEST_dual_store_replicated_cluster: pg_deep_scrub 3.0 2026-02-19T13:08:43.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_deep_scrub: local pgid=3.0 2026-02-19T13:08:43.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_deep_scrub: wait_for_pg_clean 3.0 2026-02-19T13:08:43.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=3.0 2026-02-19T13:08:43.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:08:43.644 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:08:43.644 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:08:43.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:08:43.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:08:43.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:08:43.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:08:43.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:08:43.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:08:43.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:08:43.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:08:43.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:08:44.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:08:44.115 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:08:44.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:08:44.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:44.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:08:44.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=712964571176 2026-02-19T13:08:44.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 712964571176 2026-02-19T13:08:44.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571176' 2026-02-19T13:08:44.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:44.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:08:44.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=755914244123 2026-02-19T13:08:44.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 755914244123 2026-02-19T13:08:44.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571176 1-755914244123' 2026-02-19T13:08:44.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:44.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-712964571176 2026-02-19T13:08:44.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:44.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:08:44.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-712964571176 2026-02-19T13:08:44.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:44.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=712964571176 2026-02-19T13:08:44.264 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 712964571176 2026-02-19T13:08:44.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 712964571176' 2026-02-19T13:08:44.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:44.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571174 -lt 712964571176 2026-02-19T13:08:44.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:08:45.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:08:45.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:45.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571177 -lt 712964571176 2026-02-19T13:08:45.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:45.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-755914244123 2026-02-19T13:08:45.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:45.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:08:45.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-755914244123 2026-02-19T13:08:45.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:45.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=755914244123 2026-02-19T13:08:45.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 755914244123' 2026-02-19T13:08:45.584 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 755914244123 2026-02-19T13:08:45.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:45.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 755914244124 -lt 755914244123 2026-02-19T13:08:45.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:08:45.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-19T13:08:45.756 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 3.0 loop 0 2026-02-19T13:08:45.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 3.0 2026-02-19T13:08:45.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=3.0 2026-02-19T13:08:45.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:08:45.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 3.0 query 2026-02-19T13:08:45.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:08:45.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean+inconsistent 2026-02-19T13:08:45.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean+inconsistent == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:08:45.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:08:45.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:08:45.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:08:45.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:45.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:08:45.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:45.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:08:45.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: local last_scrub=2026-02-19T13:08:39.017771+0000 2026-02-19T13:08:45.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_deep_scrub: ceph pg deep-scrub 3.0 2026-02-19T13:08:46.133 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2026-02-19T13:08:46.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_deep_scrub: wait_for_scrub 3.0 2026-02-19T13:08:39.017771+0000 last_deep_scrub_stamp 2026-02-19T13:08:46.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:08:46.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:08:39.017771+0000 2026-02-19T13:08:46.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-19T13:08:46.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:08:46.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:08:46.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:08:46.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:46.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:08:46.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:46.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:08:46.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:08:39.017771+0000 '>' 2026-02-19T13:08:39.017771+0000 2026-02-19T13:08:46.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:08:47.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:08:47.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:08:47.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-19T13:08:47.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:47.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:08:47.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-19T13:08:47.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:47.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:08:46.921568+0000 '>' 2026-02-19T13:08:39.017771+0000 2026-02-19T13:08:47.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:08:47.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6634: TEST_dual_store_replicated_cluster: (( extr_dbg >= 3 )) 2026-02-19T13:08:47.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6635: TEST_dual_store_replicated_cluster: (( extr_dbg >= 3 )) 2026-02-19T13:08:47.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6637: TEST_dual_store_replicated_cluster: (( extr_dbg >= 3 )) 2026-02-19T13:08:47.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6641: TEST_dual_store_replicated_cluster: rados list-inconsistent-obj 3.0 2026-02-19T13:08:47.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6641: TEST_dual_store_replicated_cluster: jq .inconsistents 2026-02-19T13:08:47.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6641: TEST_dual_store_replicated_cluster: jq 'def walk(f): 2026-02-19T13:08:47.470 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:08:47.470 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:08:47.471 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:08:47.471 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:08:47.471 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:08:47.471 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:08:47.471 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:08:47.471 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:08:47.471 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:08:47.471 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:08:47.471 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:08:47.471 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-19T13:08:47.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6642: TEST_dual_store_replicated_cluster: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:08:47.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6644: TEST_dual_store_replicated_cluster: rados list-inconsistent-obj 3.0 2026-02-19T13:08:47.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6644: TEST_dual_store_replicated_cluster: jq 'def walk(f): 2026-02-19T13:08:47.502 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:08:47.502 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:08:47.502 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:08:47.502 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:08:47.502 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:08:47.502 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:08:47.502 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:08:47.502 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:08:47.503 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:08:47.503 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:08:47.503 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:08:47.503 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-19T13:08:47.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6644: TEST_dual_store_replicated_cluster: jq 'del(.inconsistents[] | select(.object.name == "ROBJ13"))' 2026-02-19T13:08:47.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6645: TEST_dual_store_replicated_cluster: jq .inconsistents 2026-02-19T13:08:47.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6645: TEST_dual_store_replicated_cluster: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:08:47.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6646: TEST_dual_store_replicated_cluster: (( extr_dbg >= 3 )) 2026-02-19T13:08:47.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6649: TEST_dual_store_replicated_cluster: pg_scrub 3.0 2026-02-19T13:08:47.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=3.0 2026-02-19T13:08:47.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 3.0 2026-02-19T13:08:47.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=3.0 2026-02-19T13:08:47.540 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:08:47.541 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:08:47.541 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:08:47.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:08:47.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:08:47.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:08:47.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:08:47.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:08:47.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:08:47.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:08:47.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:08:47.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:08:48.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:08:48.017 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:08:48.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:08:48.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:48.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:08:48.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=712964571181 2026-02-19T13:08:48.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 712964571181 2026-02-19T13:08:48.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571181' 2026-02-19T13:08:48.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:08:48.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:08:48.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=755914244128 2026-02-19T13:08:48.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 755914244128 2026-02-19T13:08:48.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-712964571181 1-755914244128' 2026-02-19T13:08:48.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:48.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-712964571181 2026-02-19T13:08:48.163 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:48.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:08:48.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-712964571181 2026-02-19T13:08:48.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:48.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=712964571181 2026-02-19T13:08:48.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 712964571181' 2026-02-19T13:08:48.167 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 712964571181 2026-02-19T13:08:48.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:48.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571179 -lt 712964571181 2026-02-19T13:08:48.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:08:49.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:08:49.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:08:49.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 712964571182 -lt 712964571181 2026-02-19T13:08:49.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:08:49.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-755914244128 2026-02-19T13:08:49.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:08:49.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:08:49.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-755914244128 2026-02-19T13:08:49.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:08:49.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=755914244128 2026-02-19T13:08:49.499 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 755914244128 2026-02-19T13:08:49.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 755914244128' 2026-02-19T13:08:49.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:08:49.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 755914244129 -lt 755914244128 2026-02-19T13:08:49.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:08:49.658 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 3.0 loop 0 2026-02-19T13:08:49.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-19T13:08:49.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 3.0 2026-02-19T13:08:49.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=3.0 2026-02-19T13:08:49.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:08:49.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 3.0 query 2026-02-19T13:08:49.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:08:49.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean+inconsistent 2026-02-19T13:08:49.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean+inconsistent == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:08:49.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:08:49.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:08:49.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 3.0 2026-02-19T13:08:49.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:49.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:08:49.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:49.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:08:49.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-19T13:08:46.921568+0000 2026-02-19T13:08:49.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 3.0 2026-02-19T13:08:50.036 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 3.0 on osd.1 to scrub 2026-02-19T13:08:50.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 3.0 2026-02-19T13:08:46.921568+0000 2026-02-19T13:08:50.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=3.0 2026-02-19T13:08:50.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:08:46.921568+0000 2026-02-19T13:08:50.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:08:50.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:08:50.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:08:50.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:08:50.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:50.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:08:50.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:50.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:08:50.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:08:46.921568+0000 '>' 2026-02-19T13:08:46.921568+0000 2026-02-19T13:08:50.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:08:51.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:08:51.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:08:51.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-19T13:08:51.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=3.0 2026-02-19T13:08:51.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:08:51.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-19T13:08:51.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:08:51.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:08:50.905340+0000 '>' 2026-02-19T13:08:46.921568+0000 2026-02-19T13:08:51.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:08:51.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6651: TEST_dual_store_replicated_cluster: (( extr_dbg >= 3 )) 2026-02-19T13:08:51.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6652: TEST_dual_store_replicated_cluster: (( extr_dbg >= 3 )) 2026-02-19T13:08:51.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6654: TEST_dual_store_replicated_cluster: (( extr_dbg >= 3 )) 2026-02-19T13:08:51.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6658: TEST_dual_store_replicated_cluster: rados list-inconsistent-obj 3.0 2026-02-19T13:08:51.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6658: TEST_dual_store_replicated_cluster: jq 'def walk(f): 2026-02-19T13:08:51.365 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:08:51.365 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:08:51.365 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:08:51.365 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:08:51.365 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:08:51.365 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:08:51.365 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:08:51.366 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:08:51.366 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:08:51.366 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:08:51.366 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:08:51.366 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-19T13:08:51.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6658: TEST_dual_store_replicated_cluster: jq .inconsistents 2026-02-19T13:08:51.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6658: TEST_dual_store_replicated_cluster: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:08:51.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6660: TEST_dual_store_replicated_cluster: rados list-inconsistent-obj 3.0 2026-02-19T13:08:51.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6660: TEST_dual_store_replicated_cluster: jq 'def walk(f): 2026-02-19T13:08:51.397 INFO:tasks.workunit.client.0.trial144.stderr: . as $in 2026-02-19T13:08:51.397 INFO:tasks.workunit.client.0.trial144.stderr: | if type == "object" then 2026-02-19T13:08:51.397 INFO:tasks.workunit.client.0.trial144.stderr: reduce keys[] as $key 2026-02-19T13:08:51.397 INFO:tasks.workunit.client.0.trial144.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-19T13:08:51.397 INFO:tasks.workunit.client.0.trial144.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-19T13:08:51.398 INFO:tasks.workunit.client.0.trial144.stderr: else f 2026-02-19T13:08:51.398 INFO:tasks.workunit.client.0.trial144.stderr: end; 2026-02-19T13:08:51.398 INFO:tasks.workunit.client.0.trial144.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-19T13:08:51.398 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-19T13:08:51.398 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-19T13:08:51.398 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-19T13:08:51.398 INFO:tasks.workunit.client.0.trial144.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-19T13:08:51.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6660: TEST_dual_store_replicated_cluster: jq 'del(.inconsistents[] | select(.object.name == "ROBJ13"))' 2026-02-19T13:08:51.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6661: TEST_dual_store_replicated_cluster: jq .inconsistents 2026-02-19T13:08:51.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6661: TEST_dual_store_replicated_cluster: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-19T13:08:51.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6665: TEST_dual_store_replicated_cluster: diff -u td/osd-scrub-repair/dpPart2_w13_results.json td/osd-scrub-repair/sh2Part2_w13_results.json 2026-02-19T13:08:51.435 INFO:tasks.workunit.client.0.trial144.stdout:--- td/osd-scrub-repair/dpPart2_w13_results.json 2026-02-19 13:08:47.493660933 +0000 2026-02-19T13:08:51.435 INFO:tasks.workunit.client.0.trial144.stdout:+++ td/osd-scrub-repair/sh2Part2_w13_results.json 2026-02-19 13:08:51.389577513 +0000 2026-02-19T13:08:51.435 INFO:tasks.workunit.client.0.trial144.stdout:@@ -233,6 +233,38 @@ 2026-02-19T13:08:51.436 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:08:51.436 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head" 2026-02-19T13:08:51.436 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:08:51.436 INFO:tasks.workunit.client.0.trial144.stdout:+ "selected_object_info": { 2026-02-19T13:08:51.436 INFO:tasks.workunit.client.0.trial144.stdout:+ "alloc_hint_flags": 0, 2026-02-19T13:08:51.436 INFO:tasks.workunit.client.0.trial144.stdout:+ "data_digest": "0x2ddbf8f5", 2026-02-19T13:08:51.436 INFO:tasks.workunit.client.0.trial144.stdout:+ "expected_object_size": 0, 2026-02-19T13:08:51.436 INFO:tasks.workunit.client.0.trial144.stdout:+ "expected_write_size": 0, 2026-02-19T13:08:51.436 INFO:tasks.workunit.client.0.trial144.stdout:+ "flags": [ 2026-02-19T13:08:51.436 INFO:tasks.workunit.client.0.trial144.stdout:+ "dirty", 2026-02-19T13:08:51.436 INFO:tasks.workunit.client.0.trial144.stdout:+ "omap", 2026-02-19T13:08:51.436 INFO:tasks.workunit.client.0.trial144.stdout:+ "data_digest", 2026-02-19T13:08:51.437 INFO:tasks.workunit.client.0.trial144.stdout:+ "omap_digest" 2026-02-19T13:08:51.437 INFO:tasks.workunit.client.0.trial144.stdout:+ ], 2026-02-19T13:08:51.437 INFO:tasks.workunit.client.0.trial144.stdout:+ "lost": 0, 2026-02-19T13:08:51.437 INFO:tasks.workunit.client.0.trial144.stdout:+ "manifest": { 2026-02-19T13:08:51.437 INFO:tasks.workunit.client.0.trial144.stdout:+ "type": 0 2026-02-19T13:08:51.437 INFO:tasks.workunit.client.0.trial144.stdout:+ }, 2026-02-19T13:08:51.437 INFO:tasks.workunit.client.0.trial144.stdout:+ "oid": { 2026-02-19T13:08:51.437 INFO:tasks.workunit.client.0.trial144.stdout:+ "hash": 2682806379, 2026-02-19T13:08:51.437 INFO:tasks.workunit.client.0.trial144.stdout:+ "key": "", 2026-02-19T13:08:51.437 INFO:tasks.workunit.client.0.trial144.stdout:+ "max": 0, 2026-02-19T13:08:51.437 INFO:tasks.workunit.client.0.trial144.stdout:+ "namespace": "", 2026-02-19T13:08:51.437 INFO:tasks.workunit.client.0.trial144.stdout:+ "oid": "ROBJ13", 2026-02-19T13:08:51.437 INFO:tasks.workunit.client.0.trial144.stdout:+ "pool": 3, 2026-02-19T13:08:51.438 INFO:tasks.workunit.client.0.trial144.stdout:+ "snapid": -2 2026-02-19T13:08:51.438 INFO:tasks.workunit.client.0.trial144.stdout:+ }, 2026-02-19T13:08:51.438 INFO:tasks.workunit.client.0.trial144.stdout:+ "omap_digest": "0x6441854d", 2026-02-19T13:08:51.438 INFO:tasks.workunit.client.0.trial144.stdout:+ "shard_versions": [], 2026-02-19T13:08:51.438 INFO:tasks.workunit.client.0.trial144.stdout:+ "size": 7, 2026-02-19T13:08:51.438 INFO:tasks.workunit.client.0.trial144.stdout:+ "truncate_seq": 0, 2026-02-19T13:08:51.438 INFO:tasks.workunit.client.0.trial144.stdout:+ "truncate_size": 0, 2026-02-19T13:08:51.438 INFO:tasks.workunit.client.0.trial144.stdout:+ "user_version": 39, 2026-02-19T13:08:51.438 INFO:tasks.workunit.client.0.trial144.stdout:+ "watchers": {} 2026-02-19T13:08:51.438 INFO:tasks.workunit.client.0.trial144.stdout:+ }, 2026-02-19T13:08:51.438 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:08:51.438 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:08:51.438 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:08:51.439 INFO:tasks.workunit.client.0.trial144.stdout:@@ -242,17 +274,14 @@ 2026-02-19T13:08:51.439 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false 2026-02-19T13:08:51.439 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:08:51.439 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:08:51.439 INFO:tasks.workunit.client.0.trial144.stdout:- "errors": [ 2026-02-19T13:08:51.439 INFO:tasks.workunit.client.0.trial144.stdout:- "read_error" 2026-02-19T13:08:51.439 INFO:tasks.workunit.client.0.trial144.stdout:- ], 2026-02-19T13:08:51.439 INFO:tasks.workunit.client.0.trial144.stdout:+ "errors": [], 2026-02-19T13:08:51.439 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:08:51.439 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:08:51.439 INFO:tasks.workunit.client.0.trial144.stdout: "size": 7 2026-02-19T13:08:51.439 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:08:51.439 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:08:51.439 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [ 2026-02-19T13:08:51.440 INFO:tasks.workunit.client.0.trial144.stdout:- "stat_error", 2026-02-19T13:08:51.440 INFO:tasks.workunit.client.0.trial144.stdout:- "read_error" 2026-02-19T13:08:51.440 INFO:tasks.workunit.client.0.trial144.stdout:+ "stat_error" 2026-02-19T13:08:51.440 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:08:51.440 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:08:51.440 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:08:51.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6666: TEST_dual_store_replicated_cluster: diff -u td/osd-scrub-repair/dpPart2_wo13_results.json td/osd-scrub-repair/shPart2_wo13_results.json 2026-02-19T13:08:51.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6668: TEST_dual_store_replicated_cluster: ceph osd pool rm csr_pool csr_pool --yes-i-really-really-mean-it 2026-02-19T13:08:51.643 INFO:tasks.workunit.client.0.trial144.stderr:pool 'csr_pool' removed 2026-02-19T13:08:51.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6669: TEST_dual_store_replicated_cluster: return 0 2026-02-19T13:08:51.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T13:08:51.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:08:51.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:08:51.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:08:51.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:08:51.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:08:51.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:08:51.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:08:51.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:08:51.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:08:51.770 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:08:51.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:08:51.771 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:08:51.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:08:51.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:08:51.774 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:08:51.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:08:51.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:08:51.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:08:51.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:08:51.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:08:51.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:08:51.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:08:51.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:08:51.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:51.810 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:51.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:08:51.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:08:51.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:08:51.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T13:08:51.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T13:08:51.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T13:08:51.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T13:08:51.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:08:51.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:08:51.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:08:51.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:08:51.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:08:51.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:08:51.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:08:51.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:08:51.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:08:51.816 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:08:51.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:08:51.817 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:08:51.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:08:51.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:08:51.819 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:08:51.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:08:51.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:08:51.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:08:51.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:08:51.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:08:51.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:08:51.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:08:51.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:08:51.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:51.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:51.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:08:51.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:08:51.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:08:51.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T13:08:51.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:08:51.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:51.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:51.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T13:08:51.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:08:51.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:08:51.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:08:51.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T13:08:51.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_list_missing_erasure_coded_appends td/osd-scrub-repair 2026-02-19T13:08:51.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1037: TEST_list_missing_erasure_coded_appends: list_missing_erasure_coded td/osd-scrub-repair false 2026-02-19T13:08:51.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:975: list_missing_erasure_coded: local dir=td/osd-scrub-repair 2026-02-19T13:08:51.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:976: list_missing_erasure_coded: local allow_overwrites=false 2026-02-19T13:08:51.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:977: list_missing_erasure_coded: local poolname=ecpool 2026-02-19T13:08:51.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:979: list_missing_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-19T13:08:51.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T13:08:51.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:08:51.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:08:51.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:08:51.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T13:08:51.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T13:08:51.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:08:51.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:08:51.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:08:51.855 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:08:51.855 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:51.855 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:51.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:51.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:08:51.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:08:51.885 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:08:51.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:08:51.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:08:51.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:08:51.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:08:51.887 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:08:51.887 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:08:51.887 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:08:51.887 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:08:51.887 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:51.887 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:51.888 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:08:51.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:08:51.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T13:08:51.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:08:51.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:08:51.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:08:51.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:08:51.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:08:51.945 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:08:51.946 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:08:51.946 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:08:51.946 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:08:51.946 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:51.946 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:51.947 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:08:51.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:08:51.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T13:08:52.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:980: list_missing_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-19T13:08:52.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T13:08:52.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:08:52.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:08:52.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:08:52.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T13:08:52.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:08:52.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:08:52.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:08:52.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:08:52.103 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:08:52.103 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:52.103 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:52.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:52.104 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:08:52.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:08:52.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:981: list_missing_erasure_coded: seq 0 2 2026-02-19T13:08:52.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:981: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:08:52.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:982: list_missing_erasure_coded: run_osd td/osd-scrub-repair 0 2026-02-19T13:08:52.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:08:52.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:08:52.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:08:52.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:08:52.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:08:52.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:08:52.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:08:52.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:08:52.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:08:52.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:08:52.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:08:52.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:08:52.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:08:52.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:08:52.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:08:52.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:08:52.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:08:52.129 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:08:52.129 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:52.129 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:52.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:52.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:52.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:08:52.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:08:52.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:08:52.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:08:52.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:08:52.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:08:52.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:08:52.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:08:52.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:08:52.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:08:52.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:08:52.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:08:52.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:08:52.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=74e349f2-e64c-415b-a9ed-0c24db9d3ab6 2026-02-19T13:08:52.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 74e349f2-e64c-415b-a9ed-0c24db9d3ab6' 2026-02-19T13:08:52.131 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 74e349f2-e64c-415b-a9ed-0c24db9d3ab6 2026-02-19T13:08:52.131 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:08:52.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBkC5dp60NrCBAAf2MwohdHrTvHgRXMsvw6dg== 2026-02-19T13:08:52.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBkC5dp60NrCBAAf2MwohdHrTvHgRXMsvw6dg=="}' 2026-02-19T13:08:52.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 74e349f2-e64c-415b-a9ed-0c24db9d3ab6 -i td/osd-scrub-repair/0/new.json 2026-02-19T13:08:52.229 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:08:52.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T13:08:52.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBkC5dp60NrCBAAf2MwohdHrTvHgRXMsvw6dg== --osd-uuid 74e349f2-e64c-415b-a9ed-0c24db9d3ab6 2026-02-19T13:08:52.253 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:52.249+0000 7f49839e4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:52.255 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:52.253+0000 7f49839e4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:52.256 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:52.253+0000 7f49839e4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:52.256 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:52.253+0000 7f49839e4a40 -1 bdev(0x55d30ed42c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:08:52.256 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:52.253+0000 7f49839e4a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T13:08:53.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T13:08:53.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:08:53.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:08:53.064 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:08:53.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:08:53.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:08:53.272 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:08:53.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:08:53.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:08:53.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:08:53.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:08:53.287 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:53.281+0000 7f4e88623a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:53.288 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:53.285+0000 7f4e88623a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:53.289 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:53.285+0000 7f4e88623a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:53.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:08:53.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:08:53.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:08:53.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:08:53.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:08:53.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:53.433 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:08:53.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:08:53.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:53.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:08:53.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:08:53.644 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:53.641+0000 7f4e88623a40 -1 Falling back to public interface 2026-02-19T13:08:54.273 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:54.269+0000 7f4e88623a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:08:54.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:08:54.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:54.595 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:08:54.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:08:54.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:54.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:08:54.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:08:55.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:08:55.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:55.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:08:55.759 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:08:55.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:55.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:08:55.920 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3075147945,v1:127.0.0.1:6803/3075147945] [v2:127.0.0.1:6804/3075147945,v1:127.0.0.1:6805/3075147945] exists,up 74e349f2-e64c-415b-a9ed-0c24db9d3ab6 2026-02-19T13:08:55.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:08:55.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:08:55.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:08:55.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:981: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:08:55.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:982: list_missing_erasure_coded: run_osd td/osd-scrub-repair 1 2026-02-19T13:08:55.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:08:55.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:08:55.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:08:55.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:08:55.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:08:55.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:08:55.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:08:55.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:08:55.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:08:55.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:08:55.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:08:55.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:08:55.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:08:55.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:08:55.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:08:55.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:08:55.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:08:55.926 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:08:55.926 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:55.926 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:55.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:55.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:55.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:08:55.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:08:55.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:08:55.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:08:55.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:08:55.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:08:55.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:08:55.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:08:55.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:08:55.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:08:55.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:08:55.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:08:55.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:08:55.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c51d0207-346b-456e-9604-e920c2eeccd5 2026-02-19T13:08:55.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 c51d0207-346b-456e-9604-e920c2eeccd5' 2026-02-19T13:08:55.928 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 c51d0207-346b-456e-9604-e920c2eeccd5 2026-02-19T13:08:55.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:08:55.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBnC5dpPHfyNxAA5dvIrcWgCk+MuJ+ipZAy1A== 2026-02-19T13:08:55.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBnC5dpPHfyNxAA5dvIrcWgCk+MuJ+ipZAy1A=="}' 2026-02-19T13:08:55.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c51d0207-346b-456e-9604-e920c2eeccd5 -i td/osd-scrub-repair/1/new.json 2026-02-19T13:08:56.104 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:08:56.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T13:08:56.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBnC5dpPHfyNxAA5dvIrcWgCk+MuJ+ipZAy1A== --osd-uuid c51d0207-346b-456e-9604-e920c2eeccd5 2026-02-19T13:08:56.132 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:56.129+0000 7f6c92eeda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:56.134 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:56.129+0000 7f6c92eeda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:56.135 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:56.129+0000 7f6c92eeda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:56.135 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:56.133+0000 7f6c92eeda40 -1 bdev(0x55f041d9ec00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:08:56.136 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:56.133+0000 7f6c92eeda40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T13:08:56.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T13:08:56.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:08:56.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:08:56.791 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:08:56.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:08:56.991 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:08:56.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:08:56.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:08:56.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:08:56.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:08:56.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:08:57.007 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:57.005+0000 7f16775e7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:57.009 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:57.005+0000 7f16775e7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:57.010 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:57.005+0000 7f16775e7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:57.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:08:57.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:08:57.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:08:57.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:08:57.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:08:57.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:57.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:08:57.151 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:08:57.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:57.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:08:57.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:08:57.381 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:57.377+0000 7f16775e7a40 -1 Falling back to public interface 2026-02-19T13:08:57.953 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:57.949+0000 7f16775e7a40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:08:58.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:08:58.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:58.312 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:08:58.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:08:58.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:58.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:08:58.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:08:59.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:08:59.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:08:59.478 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:08:59.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:08:59.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:08:59.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:08:59.634 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4091388952,v1:127.0.0.1:6811/4091388952] [v2:127.0.0.1:6812/4091388952,v1:127.0.0.1:6813/4091388952] exists,up c51d0207-346b-456e-9604-e920c2eeccd5 2026-02-19T13:08:59.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:08:59.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:08:59.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:08:59.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:981: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:08:59.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:982: list_missing_erasure_coded: run_osd td/osd-scrub-repair 2 2026-02-19T13:08:59.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:08:59.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:08:59.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:08:59.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:08:59.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:08:59.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:08:59.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:08:59.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:08:59.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:08:59.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T13:08:59.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T13:08:59.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:08:59.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:08:59.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:08:59.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:08:59.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:08:59.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:08:59.639 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:08:59.639 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:08:59.640 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:08:59.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:59.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:08:59.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:08:59.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:08:59.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:08:59.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:08:59.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:08:59.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:08:59.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:08:59.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:08:59.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:08:59.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:08:59.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:08:59.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T13:08:59.641 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:08:59.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=87f2f6e0-9511-4319-8ed1-b33d82c0f4f1 2026-02-19T13:08:59.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 87f2f6e0-9511-4319-8ed1-b33d82c0f4f1' 2026-02-19T13:08:59.642 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 87f2f6e0-9511-4319-8ed1-b33d82c0f4f1 2026-02-19T13:08:59.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:08:59.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBrC5dpZG7VJhAAWwFozXA/htRIxZL51ZJXDw== 2026-02-19T13:08:59.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBrC5dpZG7VJhAAWwFozXA/htRIxZL51ZJXDw=="}' 2026-02-19T13:08:59.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 87f2f6e0-9511-4319-8ed1-b33d82c0f4f1 -i td/osd-scrub-repair/2/new.json 2026-02-19T13:08:59.806 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:08:59.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T13:08:59.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBrC5dpZG7VJhAAWwFozXA/htRIxZL51ZJXDw== --osd-uuid 87f2f6e0-9511-4319-8ed1-b33d82c0f4f1 2026-02-19T13:08:59.833 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:59.829+0000 7f26091d0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:59.835 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:59.833+0000 7f26091d0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:59.836 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:59.833+0000 7f26091d0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:08:59.836 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:59.833+0000 7f26091d0a40 -1 bdev(0x561d8419ec00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:08:59.837 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:08:59.833+0000 7f26091d0a40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T13:09:00.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T13:09:00.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:09:00.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:09:00.471 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T13:09:00.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:09:00.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:09:00.665 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T13:09:00.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:09:00.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:09:00.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:09:00.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:09:00.689 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:00.685+0000 7f4b92914a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:00.691 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:00.689+0000 7f4b92914a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:00.692 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:00.689+0000 7f4b92914a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:00.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:09:00.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:00.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:09:00.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:00.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:00.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:00.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:00.824 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:09:00.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:00.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:09:00.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:01.040 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:01.037+0000 7f4b92914a40 -1 Falling back to public interface 2026-02-19T13:09:01.586 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:01.581+0000 7f4b92914a40 -1 osd.2 0 log_to_monitors true 2026-02-19T13:09:01.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:01.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:01.983 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:09:01.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:01.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:01.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:09:02.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:03.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:03.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:03.141 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:09:03.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:09:03.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:03.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:09:03.302 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2889254564,v1:127.0.0.1:6819/2889254564] [v2:127.0.0.1:6820/2889254564,v1:127.0.0.1:6821/2889254564] exists,up 87f2f6e0-9511-4319-8ed1-b33d82c0f4f1 2026-02-19T13:09:03.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:03.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:03.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:03.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:984: list_missing_erasure_coded: create_rbd_pool 2026-02-19T13:09:03.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T13:09:03.450 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T13:09:03.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T13:09:03.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:09:03.679 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T13:09:03.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:09:04.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T13:09:04.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:985: list_missing_erasure_coded: wait_for_clean 2026-02-19T13:09:04.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:09:04.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:09:04.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:09:04.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:09:04.959 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:09:04.959 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:09:04.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:09:04.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:09:04.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:09:05.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:09:05.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:09:05.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:09:05.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:09:05.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:09:05.080 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:09:05.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:09:05.239 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:09:05.239 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:09:05.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:09:05.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:05.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:09:05.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T13:09:05.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T13:09:05.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T13:09:05.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:05.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:09:05.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-19T13:09:05.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-19T13:09:05.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705667' 2026-02-19T13:09:05.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:05.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:09:05.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542146 2026-02-19T13:09:05.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542146 2026-02-19T13:09:05.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705667 2-60129542146' 2026-02-19T13:09:05.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:05.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T13:09:05.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:05.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:09:05.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T13:09:05.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:05.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T13:09:05.463 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T13:09:05.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T13:09:05.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:09:05.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T13:09:05.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:09:06.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:09:06.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:09:06.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836483 2026-02-19T13:09:06.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:06.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-19T13:09:06.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:06.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:09:06.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705667 2026-02-19T13:09:06.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:06.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705667 2026-02-19T13:09:06.796 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705667 2026-02-19T13:09:06.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-19T13:09:06.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:09:06.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705667 -lt 38654705667 2026-02-19T13:09:06.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:06.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542146 2026-02-19T13:09:06.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:06.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:09:06.967 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542146 2026-02-19T13:09:06.967 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:06.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542146 2026-02-19T13:09:06.969 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542146 2026-02-19T13:09:06.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-19T13:09:06.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:09:07.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542146 -lt 60129542146 2026-02-19T13:09:07.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:09:07.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:09:07.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:09:07.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T13:09:07.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:09:07.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:09:07.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:09:07.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:09:07.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:09:07.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:09:07.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:09:07.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T13:09:07.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:09:07.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:09:07.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:09:07.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T13:09:07.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:09:07.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:09:07.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:987: list_missing_erasure_coded: create_ec_pool ecpool false k=2 m=1 2026-02-19T13:09:07.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: create_ec_pool: local pool_name=ecpool 2026-02-19T13:09:07.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: create_ec_pool: shift 2026-02-19T13:09:07.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: create_ec_pool: local allow_overwrites=false 2026-02-19T13:09:07.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2525: create_ec_pool: shift 2026-02-19T13:09:07.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-19T13:09:07.895 INFO:tasks.workunit.client.0.trial144.stderr:load: isa 2026-02-19T13:09:07.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-19T13:09:07.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-19T13:09:08.200 INFO:tasks.workunit.client.0.trial144.stderr:pool 'ecpool' created 2026-02-19T13:09:08.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:09:09.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: create_ec_pool: '[' false = true ']' 2026-02-19T13:09:09.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2535: create_ec_pool: wait_for_clean 2026-02-19T13:09:09.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:09:09.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:09:09.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:09:09.216 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:09:09.217 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:09:09.217 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:09:09.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:09:09.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:09:09.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:09:09.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:09:09.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:09:09.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:09:09.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:09:09.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:09:09.339 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:09:09.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:09:09.497 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:09:09.497 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:09:09.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:09:09.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:09.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:09:09.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-19T13:09:09.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-19T13:09:09.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485' 2026-02-19T13:09:09.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:09.571 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:09:09.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705669 2026-02-19T13:09:09.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705669 2026-02-19T13:09:09.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-38654705669' 2026-02-19T13:09:09.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:09.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:09:09.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542148 2026-02-19T13:09:09.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542148 2026-02-19T13:09:09.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-38654705669 2-60129542148' 2026-02-19T13:09:09.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:09.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-19T13:09:09.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:09.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:09:09.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836485 2026-02-19T13:09:09.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:09.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836485 2026-02-19T13:09:09.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-19T13:09:09.720 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836485 2026-02-19T13:09:09.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:09:09.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-19T13:09:09.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:09:10.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:09:10.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:09:11.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-19T13:09:11.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:11.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705669 2026-02-19T13:09:11.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:11.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:09:11.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705669 2026-02-19T13:09:11.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:11.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705669 2026-02-19T13:09:11.046 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705669 2026-02-19T13:09:11.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-19T13:09:11.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:09:11.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-19T13:09:11.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:11.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542148 2026-02-19T13:09:11.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:11.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:09:11.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542148 2026-02-19T13:09:11.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:11.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542148 2026-02-19T13:09:11.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-19T13:09:11.210 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542148 2026-02-19T13:09:11.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:09:11.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-19T13:09:11.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:09:11.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:09:11.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:09:11.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:09:11.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:09:11.568 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:09:11.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:09:11.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:09:11.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:09:11.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:09:11.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:09:11.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:09:11.729 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:09:11.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:09:11.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:09:11.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:09:11.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:09:11.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:09:11.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2536: create_ec_pool: return 0 2026-02-19T13:09:11.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:990: list_missing_erasure_coded: add_something td/osd-scrub-repair ecpool MOBJ0 2026-02-19T13:09:11.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:09:11.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T13:09:11.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=MOBJ0 2026-02-19T13:09:11.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:09:11.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:09:11.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:09:12.123 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:09:12.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:09:12.328 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:09:12.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:09:12.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:09:12.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put MOBJ0 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:09:12.365 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:991: list_missing_erasure_coded: get_osds ecpool MOBJ0 2026-02-19T13:09:12.365 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T13:09:12.365 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=MOBJ0 2026-02-19T13:09:12.366 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool MOBJ0 2026-02-19T13:09:12.366 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:09:12.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-19T13:09:12.529 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:09:12.529 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:09:12.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 1 0 2026-02-19T13:09:12.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:991: list_missing_erasure_coded: osds0=('2' '1' '0') 2026-02-19T13:09:12.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:991: list_missing_erasure_coded: local -a osds0 2026-02-19T13:09:12.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:994: list_missing_erasure_coded: add_something td/osd-scrub-repair ecpool MOBJ1 2026-02-19T13:09:12.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:09:12.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T13:09:12.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=MOBJ1 2026-02-19T13:09:12.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:09:12.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:09:12.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:09:12.730 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:09:12.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:09:12.934 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:09:12.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:09:12.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:09:12.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put MOBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:09:12.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:995: list_missing_erasure_coded: get_osds ecpool MOBJ1 2026-02-19T13:09:12.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T13:09:12.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=MOBJ1 2026-02-19T13:09:12.974 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool MOBJ1 2026-02-19T13:09:12.974 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:09:13.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-19T13:09:13.134 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:09:13.135 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:09:13.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 1 0 2026-02-19T13:09:13.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:995: list_missing_erasure_coded: osds1=('2' '1' '0') 2026-02-19T13:09:13.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:995: list_missing_erasure_coded: local -a osds1 2026-02-19T13:09:13.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:998: list_missing_erasure_coded: seq 0 2 2026-02-19T13:09:13.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:998: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:09:13.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:999: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:09:13.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:13.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:13.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:13.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:13.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:13.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:13.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:998: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:09:13.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:999: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:09:13.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:13.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:13.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:13.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:13.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:13.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:13.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:998: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:09:13.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:999: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-19T13:09:13.760 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:13.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:13.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:13.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:13.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:14.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:14.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1002: list_missing_erasure_coded: id=2 2026-02-19T13:09:14.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1003: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 MOBJ0 remove 2026-02-19T13:09:14.243 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#2:a9030505:::MOBJ0:head# 2026-02-19T13:09:14.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1005: list_missing_erasure_coded: id=1 2026-02-19T13:09:14.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 MOBJ0 remove 2026-02-19T13:09:14.526 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#2:a9030505:::MOBJ0:head# 2026-02-19T13:09:14.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1009: list_missing_erasure_coded: id=1 2026-02-19T13:09:14.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 MOBJ1 remove 2026-02-19T13:09:14.817 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#2:8dd16d55:::MOBJ1:head# 2026-02-19T13:09:14.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1012: list_missing_erasure_coded: id=0 2026-02-19T13:09:14.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1013: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 MOBJ1 remove 2026-02-19T13:09:15.107 INFO:tasks.workunit.client.0.trial144.stdout:remove 2#2:8dd16d55:::MOBJ1:head# 2026-02-19T13:09:15.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1016: list_missing_erasure_coded: seq 0 2 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1016: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1017: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 0 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:09:15.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:09:15.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:15.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:15.198 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:15.198 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:15.198 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:15.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:15.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:15.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:15.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:09:15.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:09:15.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:15.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:15.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:15.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:15.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:09:15.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:09:15.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:09:15.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:09:15.200 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:09:15.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:09:15.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:09:15.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:09:15.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:09:15.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:09:15.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:09:15.217 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:15.213+0000 7f9075b3fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:15.219 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:15.217+0000 7f9075b3fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:15.220 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:15.217+0000 7f9075b3fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:15.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:09:15.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:15.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:09:15.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:15.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:15.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:15.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:15.364 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:09:15.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:15.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:15.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:15.597 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:15.593+0000 7f9075b3fa40 -1 Falling back to public interface 2026-02-19T13:09:16.234 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:16.229+0000 7f9075b3fa40 -1 osd.0 32 log_to_monitors true 2026-02-19T13:09:16.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:16.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:16.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:16.520 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:09:16.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:16.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:16.680 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 39 up_thru 39 down_at 33 last_clean_interval [5,32) [v2:127.0.0.1:6802/77441768,v1:127.0.0.1:6803/77441768] [v2:127.0.0.1:6804/77441768,v1:127.0.0.1:6805/77441768] exists,up 74e349f2-e64c-415b-a9ed-0c24db9d3ab6 2026-02-19T13:09:16.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:16.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:16.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:16.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1016: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:09:16.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1017: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 1 2026-02-19T13:09:16.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:09:16.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:09:16.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:09:16.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:09:16.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:09:16.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:09:16.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:16.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:16.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:16.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:09:16.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:09:16.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:16.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:09:16.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:09:16.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:09:16.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:16.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:16.682 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:16.682 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:16.682 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:16.682 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:16.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:16.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:16.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:09:16.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:09:16.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:16.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:16.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:16.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:16.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:09:16.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:09:16.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:09:16.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:09:16.683 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:09:16.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:09:16.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:09:16.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:09:16.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:09:16.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:09:16.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:09:16.700 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:16.697+0000 7fb4a7b71a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:16.702 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:16.697+0000 7fb4a7b71a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:16.702 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:16.697+0000 7fb4a7b71a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:16.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:09:16.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:16.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:09:16.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:16.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:16.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:16.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:16.847 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:09:16.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:16.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:09:17.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:17.081 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:17.077+0000 7fb4a7b71a40 -1 Falling back to public interface 2026-02-19T13:09:17.753 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:17.749+0000 7fb4a7b71a40 -1 osd.1 34 log_to_monitors true 2026-02-19T13:09:18.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:18.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:18.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:18.009 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:09:18.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:18.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:09:18.165 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 43 up_thru 43 down_at 35 last_clean_interval [9,34) [v2:127.0.0.1:6810/1954981401,v1:127.0.0.1:6811/1954981401] [v2:127.0.0.1:6812/1954981401,v1:127.0.0.1:6813/1954981401] exists,up c51d0207-346b-456e-9604-e920c2eeccd5 2026-02-19T13:09:18.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:18.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:18.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:18.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1016: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:09:18.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1017: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 2 2026-02-19T13:09:18.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:09:18.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:09:18.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:09:18.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:09:18.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:09:18.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:09:18.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:18.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:18.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:18.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T13:09:18.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T13:09:18.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:18.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:09:18.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:09:18.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:09:18.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:18.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:18.168 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:18.168 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:18.168 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:18.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:18.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:18.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:18.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:09:18.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:09:18.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:18.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:18.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:18.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:18.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:09:18.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:09:18.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T13:09:18.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:09:18.171 INFO:tasks.workunit.client.0.trial144.stderr:start osd.2 2026-02-19T13:09:18.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:09:18.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-19T13:09:18.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:09:18.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:09:18.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:09:18.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:09:18.192 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:18.189+0000 7f6673c0aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:18.193 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:18.189+0000 7f6673c0aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:18.194 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:18.189+0000 7f6673c0aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:18.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:09:18.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:18.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:09:18.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:18.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:18.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:18.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:18.330 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:09:18.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:18.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:09:18.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:18.645 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:18.640+0000 7f6673c0aa40 -1 Falling back to public interface 2026-02-19T13:09:19.371 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:19.364+0000 7f6673c0aa40 -1 osd.2 36 log_to_monitors true 2026-02-19T13:09:19.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:19.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:19.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:19.493 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:09:19.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:19.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:09:19.650 INFO:tasks.workunit.client.0.trial144.stderr:osd.2 up in weight 1 up_from 47 up_thru 47 down_at 37 last_clean_interval [14,36) [v2:127.0.0.1:6818/371982495,v1:127.0.0.1:6819/371982495] [v2:127.0.0.1:6820/371982495,v1:127.0.0.1:6821/371982495] exists,up 87f2f6e0-9511-4319-8ed1-b33d82c0f4f1 2026-02-19T13:09:19.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:19.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:19.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:19.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1019: list_missing_erasure_coded: create_rbd_pool 2026-02-19T13:09:19.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T13:09:19.833 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' removed 2026-02-19T13:09:19.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T13:09:19.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:09:20.036 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T13:09:20.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:09:21.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T13:09:21.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1020: list_missing_erasure_coded: wait_for_clean 2026-02-19T13:09:21.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:09:21.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:09:21.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:09:21.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:09:21.333 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:09:21.333 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:09:21.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:09:21.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:09:21.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:09:21.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:09:21.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:09:21.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:09:21.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:09:21.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:09:21.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:09:21.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:09:21.610 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:09:21.610 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:09:21.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:09:21.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:21.611 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:09:21.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724547 2026-02-19T13:09:21.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724547 2026-02-19T13:09:21.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724547' 2026-02-19T13:09:21.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:21.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:09:21.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593730 2026-02-19T13:09:21.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593730 2026-02-19T13:09:21.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724547 1-184683593730' 2026-02-19T13:09:21.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:21.759 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:09:21.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462914 2026-02-19T13:09:21.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462914 2026-02-19T13:09:21.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-167503724547 1-184683593730 2-201863462914' 2026-02-19T13:09:21.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:21.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-167503724547 2026-02-19T13:09:21.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:21.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:09:21.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-167503724547 2026-02-19T13:09:21.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:21.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724547 2026-02-19T13:09:21.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 167503724547' 2026-02-19T13:09:21.836 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 167503724547 2026-02-19T13:09:21.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:09:21.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 167503724547 2026-02-19T13:09:21.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:09:22.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:09:22.995 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:09:23.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-19T13:09:23.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:23.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-184683593730 2026-02-19T13:09:23.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:23.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:09:23.156 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-184683593730 2026-02-19T13:09:23.156 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:23.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593730 2026-02-19T13:09:23.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 184683593730' 2026-02-19T13:09:23.159 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 184683593730 2026-02-19T13:09:23.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:09:23.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593730 -lt 184683593730 2026-02-19T13:09:23.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:23.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-201863462914 2026-02-19T13:09:23.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:23.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:09:23.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-201863462914 2026-02-19T13:09:23.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:23.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=201863462914 2026-02-19T13:09:23.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 201863462914' 2026-02-19T13:09:23.322 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 201863462914 2026-02-19T13:09:23.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:09:23.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 201863462914 -lt 201863462914 2026-02-19T13:09:23.494 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:09:23.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:09:23.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:09:23.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:09:23.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:09:23.691 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:09:23.691 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:09:23.691 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:09:23.691 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:09:23.691 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:09:23.691 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:09:23.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:09:23.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:09:23.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:09:23.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:09:24.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:09:24.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:09:24.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:09:24.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1023: list_missing_erasure_coded: get_pg ecpool MOBJ0 2026-02-19T13:09:24.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T13:09:24.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=MOBJ0 2026-02-19T13:09:24.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool MOBJ0 2026-02-19T13:09:24.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:09:24.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1023: list_missing_erasure_coded: local pg=2.0 2026-02-19T13:09:24.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1027: list_missing_erasure_coded: repair 2.0 2026-02-19T13:09:24.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T13:09:24.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T13:09:24.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T13:09:24.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:09:24.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:09:24.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T13:09:24.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T13:09:08.200246+0000 2026-02-19T13:09:24.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T13:09:24.496 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.2 to repair 2026-02-19T13:09:24.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T13:09:08.200246+0000 2026-02-19T13:09:24.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T13:09:24.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:09:08.200246+0000 2026-02-19T13:09:24.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:09:24.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:09:24.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:09:24.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T13:09:24.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T13:09:24.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:09:24.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:09:24.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T13:09:24.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:09:08.200246+0000 '>' 2026-02-19T13:09:08.200246+0000 2026-02-19T13:09:24.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:09:25.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:09:25.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:09:25.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T13:09:25.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T13:09:25.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:09:25.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:09:25.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T13:09:25.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:09:08.200246+0000 '>' 2026-02-19T13:09:08.200246+0000 2026-02-19T13:09:25.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:09:26.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:09:26.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:09:26.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T13:09:26.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T13:09:26.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:09:26.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:09:26.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T13:09:26.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:09:08.200246+0000 '>' 2026-02-19T13:09:08.200246+0000 2026-02-19T13:09:26.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:09:27.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:09:27.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:09:27.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T13:09:27.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T13:09:27.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:09:27.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T13:09:27.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:09:28.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:09:08.200246+0000 '>' 2026-02-19T13:09:08.200246+0000 2026-02-19T13:09:28.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:09:29.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:09:29.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:09:29.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T13:09:29.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T13:09:29.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:09:29.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:09:29.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T13:09:29.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:09:08.200246+0000 '>' 2026-02-19T13:09:08.200246+0000 2026-02-19T13:09:29.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:09:30.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:09:30.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:09:30.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T13:09:30.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T13:09:30.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:09:30.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:09:30.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T13:09:30.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:09:08.200246+0000 '>' 2026-02-19T13:09:08.200246+0000 2026-02-19T13:09:30.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:09:31.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:09:31.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:09:31.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T13:09:31.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T13:09:31.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:09:31.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T13:09:31.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:09:31.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:09:25.421151+0000 '>' 2026-02-19T13:09:08.200246+0000 2026-02-19T13:09:31.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:09:31.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: list_missing_erasure_coded: seq 0 120 2026-02-19T13:09:31.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: list_missing_erasure_coded: for i in $(seq 0 120) 2026-02-19T13:09:31.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: list_missing_erasure_coded: '[' 0 -lt 60 ']' 2026-02-19T13:09:31.641 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1031: list_missing_erasure_coded: ceph pg 2.0 list_unfound 2026-02-19T13:09:31.641 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1031: list_missing_erasure_coded: egrep 'MOBJ0|MOBJ1' 2026-02-19T13:09:31.641 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1031: list_missing_erasure_coded: wc -l 2026-02-19T13:09:31.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1031: list_missing_erasure_coded: matches=2 2026-02-19T13:09:31.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1032: list_missing_erasure_coded: '[' 2 -eq 2 ']' 2026-02-19T13:09:31.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1032: list_missing_erasure_coded: break 2026-02-19T13:09:31.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T13:09:31.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:09:31.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:09:31.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:09:31.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:31.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:31.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:31.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:31.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:31.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:31.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:09:31.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:09:31.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:09:31.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:09:31.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:09:31.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:09:31.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:31.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:09:31.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:09:31.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:31.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:09:31.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:09:31.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:09:31.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:09:31.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:31.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:31.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:09:31.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:09:31.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:09:31.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T13:09:31.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T13:09:31.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T13:09:31.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T13:09:31.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:09:31.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:09:31.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:09:31.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:31.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:31.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:31.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:31.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:31.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:31.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:09:31.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:09:31.858 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:09:31.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:09:31.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:09:31.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:09:31.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:31.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:09:31.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:09:31.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:31.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:09:31.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:09:31.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:09:31.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:09:31.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:31.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:31.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:09:31.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:09:31.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:09:31.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T13:09:31.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:09:31.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:31.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:31.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T13:09:31.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:09:31.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:09:31.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:09:31.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T13:09:31.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_list_missing_erasure_coded_overwrites td/osd-scrub-repair 2026-02-19T13:09:31.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1041: TEST_list_missing_erasure_coded_overwrites: '[' true = true ']' 2026-02-19T13:09:31.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1042: TEST_list_missing_erasure_coded_overwrites: list_missing_erasure_coded td/osd-scrub-repair true 2026-02-19T13:09:31.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:975: list_missing_erasure_coded: local dir=td/osd-scrub-repair 2026-02-19T13:09:31.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:976: list_missing_erasure_coded: local allow_overwrites=true 2026-02-19T13:09:31.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:977: list_missing_erasure_coded: local poolname=ecpool 2026-02-19T13:09:31.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:979: list_missing_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-19T13:09:31.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T13:09:31.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:09:31.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:09:31.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:09:31.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T13:09:31.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T13:09:31.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:09:31.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:31.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:31.896 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:31.896 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:31.896 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:31.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:31.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:09:31.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:09:31.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:09:31.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:09:31.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:09:31.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:09:31.930 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:09:31.930 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:09:31.930 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:31.930 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:31.930 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:31.931 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:31.931 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:31.931 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:09:31.931 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:09:31.931 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T13:09:31.987 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:09:31.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:09:31.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:09:31.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:09:31.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:09:31.989 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:09:31.989 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:31.989 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:31.989 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:31.989 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:31.990 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:31.990 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:09:31.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:09:31.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T13:09:32.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:980: list_missing_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-19T13:09:32.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T13:09:32.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:09:32.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:09:32.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:09:32.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T13:09:32.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:09:32.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:09:32.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:32.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:32.146 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:32.146 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:32.146 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:32.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:32.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:09:32.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:09:32.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:981: list_missing_erasure_coded: seq 0 2 2026-02-19T13:09:32.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:981: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:09:32.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:982: list_missing_erasure_coded: run_osd td/osd-scrub-repair 0 2026-02-19T13:09:32.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:09:32.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:09:32.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:09:32.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:09:32.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:09:32.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:09:32.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:32.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:32.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:32.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:09:32.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:09:32.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:32.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:09:32.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:09:32.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:09:32.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:32.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:32.170 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:32.170 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:32.170 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:32.172 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:32.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:32.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:32.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:09:32.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:09:32.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:09:32.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:09:32.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:32.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:32.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:32.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:32.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:09:32.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:09:32.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:09:32.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:09:32.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7e0847bc-b958-47d0-8d0f-ddbdc4b2b0b0 2026-02-19T13:09:32.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 7e0847bc-b958-47d0-8d0f-ddbdc4b2b0b0' 2026-02-19T13:09:32.173 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 7e0847bc-b958-47d0-8d0f-ddbdc4b2b0b0 2026-02-19T13:09:32.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:09:32.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCMC5dpSRr9ChAAS8mHhXchEdhLAQ2YxTb1ew== 2026-02-19T13:09:32.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCMC5dpSRr9ChAAS8mHhXchEdhLAQ2YxTb1ew=="}' 2026-02-19T13:09:32.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7e0847bc-b958-47d0-8d0f-ddbdc4b2b0b0 -i td/osd-scrub-repair/0/new.json 2026-02-19T13:09:32.273 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:09:32.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T13:09:32.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCMC5dpSRr9ChAAS8mHhXchEdhLAQ2YxTb1ew== --osd-uuid 7e0847bc-b958-47d0-8d0f-ddbdc4b2b0b0 2026-02-19T13:09:32.298 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:32.292+0000 7fb8838e7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:32.300 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:32.296+0000 7fb8838e7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:32.300 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:32.296+0000 7fb8838e7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:32.301 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:32.296+0000 7fb8838e7a40 -1 bdev(0x55a4b8536c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:09:32.301 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:32.296+0000 7fb8838e7a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T13:09:33.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T13:09:33.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:09:33.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:09:33.068 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:09:33.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:09:33.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:09:33.272 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:09:33.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:09:33.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:09:33.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:09:33.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:09:33.288 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:33.284+0000 7fa9aa733a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:33.290 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:33.284+0000 7fa9aa733a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:33.291 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:33.284+0000 7fa9aa733a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:33.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:09:33.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:33.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:09:33.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:33.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:33.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:33.430 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:09:33.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:33.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:33.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:33.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:33.689 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:33.684+0000 7fa9aa733a40 -1 Falling back to public interface 2026-02-19T13:09:34.293 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:34.288+0000 7fa9aa733a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:09:34.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:34.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:34.596 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:09:34.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:34.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:34.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:34.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:35.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:35.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:35.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:09:35.760 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:09:35.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:35.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:35.921 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2198239075,v1:127.0.0.1:6803/2198239075] [v2:127.0.0.1:6804/2198239075,v1:127.0.0.1:6805/2198239075] exists,up 7e0847bc-b958-47d0-8d0f-ddbdc4b2b0b0 2026-02-19T13:09:35.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:35.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:35.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:35.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:981: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:09:35.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:982: list_missing_erasure_coded: run_osd td/osd-scrub-repair 1 2026-02-19T13:09:35.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:09:35.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:09:35.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:09:35.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:35.922 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:35.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:35.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:09:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:09:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:09:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:09:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:09:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:09:35.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:09:35.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:09:35.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=48c6f101-a677-4564-a05f-8b44ac088736 2026-02-19T13:09:35.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 48c6f101-a677-4564-a05f-8b44ac088736' 2026-02-19T13:09:35.926 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 48c6f101-a677-4564-a05f-8b44ac088736 2026-02-19T13:09:35.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:09:35.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCPC5dpq+j5NxAAUC0AxlGq0Qswmo5MGBkpUg== 2026-02-19T13:09:35.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCPC5dpq+j5NxAAUC0AxlGq0Qswmo5MGBkpUg=="}' 2026-02-19T13:09:35.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 48c6f101-a677-4564-a05f-8b44ac088736 -i td/osd-scrub-repair/1/new.json 2026-02-19T13:09:36.101 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:09:36.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T13:09:36.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCPC5dpq+j5NxAAUC0AxlGq0Qswmo5MGBkpUg== --osd-uuid 48c6f101-a677-4564-a05f-8b44ac088736 2026-02-19T13:09:36.129 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:36.124+0000 7f5faa0e4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:36.130 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:36.124+0000 7f5faa0e4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:36.131 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:36.128+0000 7f5faa0e4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:36.132 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:36.128+0000 7f5faa0e4a40 -1 bdev(0x55678f090c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:09:36.132 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:36.128+0000 7f5faa0e4a40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T13:09:36.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T13:09:36.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:09:36.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:09:36.755 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:09:36.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:09:36.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:09:36.959 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:09:36.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:09:36.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:09:36.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:09:36.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:09:36.984 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:36.980+0000 7f21ee4bfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:36.985 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:36.980+0000 7f21ee4bfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:36.986 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:36.980+0000 7f21ee4bfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:37.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:09:37.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:37.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:09:37.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:37.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:37.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:37.119 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:09:37.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:37.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:37.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:09:37.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:37.441 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:37.436+0000 7f21ee4bfa40 -1 Falling back to public interface 2026-02-19T13:09:37.965 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:37.960+0000 7f21ee4bfa40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:09:38.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:38.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:38.277 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:09:38.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:38.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:38.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:09:38.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:39.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:39.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:39.442 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:09:39.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:09:39.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:39.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:09:39.598 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/495697386,v1:127.0.0.1:6811/495697386] [v2:127.0.0.1:6812/495697386,v1:127.0.0.1:6813/495697386] exists,up 48c6f101-a677-4564-a05f-8b44ac088736 2026-02-19T13:09:39.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:39.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:39.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:39.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:981: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:09:39.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:982: list_missing_erasure_coded: run_osd td/osd-scrub-repair 2 2026-02-19T13:09:39.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:09:39.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:09:39.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:09:39.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:09:39.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:09:39.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:09:39.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:39.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:39.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:39.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T13:09:39.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T13:09:39.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:39.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:09:39.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:09:39.600 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:09:39.600 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:39.600 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:39.602 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:39.602 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:39.602 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:39.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:39.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:39.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:39.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:09:39.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:09:39.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:09:39.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:09:39.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:39.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:39.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:39.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:39.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:09:39.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:09:39.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T13:09:39.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:09:39.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c1c53704-95e8-412b-909a-f305b6a18309 2026-02-19T13:09:39.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 c1c53704-95e8-412b-909a-f305b6a18309' 2026-02-19T13:09:39.605 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 c1c53704-95e8-412b-909a-f305b6a18309 2026-02-19T13:09:39.605 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:09:39.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCTC5dpH5DEJBAArWskejSshFl5noj3V/okpQ== 2026-02-19T13:09:39.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCTC5dpH5DEJBAArWskejSshFl5noj3V/okpQ=="}' 2026-02-19T13:09:39.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c1c53704-95e8-412b-909a-f305b6a18309 -i td/osd-scrub-repair/2/new.json 2026-02-19T13:09:39.779 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:09:39.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T13:09:39.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCTC5dpH5DEJBAArWskejSshFl5noj3V/okpQ== --osd-uuid c1c53704-95e8-412b-909a-f305b6a18309 2026-02-19T13:09:39.809 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:39.804+0000 7f266257ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:39.810 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:39.804+0000 7f266257ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:39.811 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:39.808+0000 7f266257ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:39.812 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:39.808+0000 7f266257ba40 -1 bdev(0x55b6a77a4c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:09:39.812 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:39.808+0000 7f266257ba40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T13:09:40.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T13:09:40.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:09:40.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:09:40.451 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T13:09:40.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:09:40.655 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T13:09:40.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:09:40.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:09:40.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:09:40.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:09:40.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:09:40.672 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:40.668+0000 7fe05c9a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:40.673 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:40.668+0000 7fe05c9a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:40.674 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:40.668+0000 7fe05c9a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:40.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:09:40.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:40.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:09:40.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:40.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:40.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:40.817 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:09:40.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:40.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:40.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:09:40.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:41.124 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:41.120+0000 7fe05c9a4a40 -1 Falling back to public interface 2026-02-19T13:09:41.719 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:41.712+0000 7fe05c9a4a40 -1 osd.2 0 log_to_monitors true 2026-02-19T13:09:41.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:41.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:41.976 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:09:41.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:41.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:41.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:09:42.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:43.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:43.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:43.138 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:09:43.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:09:43.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:43.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:09:43.293 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/784490207,v1:127.0.0.1:6819/784490207] [v2:127.0.0.1:6820/784490207,v1:127.0.0.1:6821/784490207] exists,up c1c53704-95e8-412b-909a-f305b6a18309 2026-02-19T13:09:43.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:43.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:43.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:43.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:984: list_missing_erasure_coded: create_rbd_pool 2026-02-19T13:09:43.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T13:09:43.441 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T13:09:43.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T13:09:43.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:09:43.669 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T13:09:43.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:09:44.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T13:09:44.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:985: list_missing_erasure_coded: wait_for_clean 2026-02-19T13:09:44.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:09:44.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:09:44.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:09:44.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:09:44.998 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:09:44.998 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:09:44.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:09:44.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:09:44.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:09:45.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:09:45.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:09:45.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:09:45.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:09:45.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:09:45.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:09:45.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:09:45.281 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:09:45.281 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:09:45.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:09:45.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:45.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:09:45.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T13:09:45.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T13:09:45.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T13:09:45.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:45.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:09:45.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705668 2026-02-19T13:09:45.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705668 2026-02-19T13:09:45.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705668' 2026-02-19T13:09:45.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:45.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:09:45.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574850 2026-02-19T13:09:45.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574850 2026-02-19T13:09:45.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705668 2-55834574850' 2026-02-19T13:09:45.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:45.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T13:09:45.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:45.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:09:45.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T13:09:45.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:45.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T13:09:45.502 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T13:09:45.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T13:09:45.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:09:45.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T13:09:45.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:09:46.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:09:46.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:09:46.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836483 2026-02-19T13:09:46.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:46.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705668 2026-02-19T13:09:46.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:46.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:09:46.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705668 2026-02-19T13:09:46.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:46.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705668 2026-02-19T13:09:46.827 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705668 2026-02-19T13:09:46.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-19T13:09:46.827 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:09:46.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-19T13:09:46.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:46.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574850 2026-02-19T13:09:46.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:47.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:09:47.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574850 2026-02-19T13:09:47.001 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:47.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574850 2026-02-19T13:09:47.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574850' 2026-02-19T13:09:47.002 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574850 2026-02-19T13:09:47.003 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:09:47.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574850 -lt 55834574850 2026-02-19T13:09:47.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:09:47.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:09:47.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:09:47.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T13:09:47.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:09:47.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:09:47.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:09:47.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:09:47.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:09:47.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:09:47.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:09:47.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T13:09:47.514 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:09:47.514 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:09:47.514 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:09:47.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T13:09:47.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:09:47.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:09:47.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:987: list_missing_erasure_coded: create_ec_pool ecpool true k=2 m=1 2026-02-19T13:09:47.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: create_ec_pool: local pool_name=ecpool 2026-02-19T13:09:47.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: create_ec_pool: shift 2026-02-19T13:09:47.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: create_ec_pool: local allow_overwrites=true 2026-02-19T13:09:47.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2525: create_ec_pool: shift 2026-02-19T13:09:47.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-19T13:09:47.906 INFO:tasks.workunit.client.0.trial144.stderr:load: isa 2026-02-19T13:09:47.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-19T13:09:47.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-19T13:09:48.211 INFO:tasks.workunit.client.0.trial144.stderr:pool 'ecpool' created 2026-02-19T13:09:48.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:09:49.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: create_ec_pool: '[' true = true ']' 2026-02-19T13:09:49.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2532: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-19T13:09:49.423 INFO:tasks.workunit.client.0.trial144.stderr:set pool 2 allow_ec_overwrites to true 2026-02-19T13:09:49.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2535: create_ec_pool: wait_for_clean 2026-02-19T13:09:49.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:09:49.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:09:49.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:09:49.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:09:49.440 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:09:49.440 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:09:49.441 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:09:49.441 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:09:49.441 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:09:49.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:09:49.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:09:49.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:09:49.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:09:49.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:09:49.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:09:49.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:09:49.736 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:09:49.736 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:09:49.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:09:49.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:49.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:09:49.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-19T13:09:49.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-19T13:09:49.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485' 2026-02-19T13:09:49.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:49.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:09:49.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705670 2026-02-19T13:09:49.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705670 2026-02-19T13:09:49.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-38654705670' 2026-02-19T13:09:49.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:49.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:09:49.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574852 2026-02-19T13:09:49.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574852 2026-02-19T13:09:49.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-38654705670 2-55834574852' 2026-02-19T13:09:49.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:49.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-19T13:09:49.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:49.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:09:49.962 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836485 2026-02-19T13:09:49.962 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:49.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836485 2026-02-19T13:09:49.964 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836485 2026-02-19T13:09:49.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-19T13:09:49.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:09:50.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-19T13:09:50.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:09:51.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:09:51.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:09:51.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-19T13:09:51.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:51.292 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705670 2026-02-19T13:09:51.292 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:51.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:09:51.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705670 2026-02-19T13:09:51.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:51.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705670 2026-02-19T13:09:51.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-19T13:09:51.297 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705670 2026-02-19T13:09:51.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:09:51.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705670 -lt 38654705670 2026-02-19T13:09:51.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:51.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574852 2026-02-19T13:09:51.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:51.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:09:51.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574852 2026-02-19T13:09:51.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:51.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574852 2026-02-19T13:09:51.460 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574852 2026-02-19T13:09:51.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574852' 2026-02-19T13:09:51.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:09:51.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-19T13:09:51.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:09:51.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:09:51.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:09:51.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:09:51.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:09:51.817 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:09:51.817 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:09:51.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:09:51.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:09:51.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:09:51.818 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:09:51.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:09:51.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:09:51.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:09:51.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:09:52.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:09:52.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:09:52.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:09:52.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2536: create_ec_pool: return 0 2026-02-19T13:09:52.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:990: list_missing_erasure_coded: add_something td/osd-scrub-repair ecpool MOBJ0 2026-02-19T13:09:52.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:09:52.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T13:09:52.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=MOBJ0 2026-02-19T13:09:52.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:09:52.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:09:52.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:09:52.376 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:09:52.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:09:52.580 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:09:52.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:09:52.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:09:52.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put MOBJ0 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:09:52.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:991: list_missing_erasure_coded: get_osds ecpool MOBJ0 2026-02-19T13:09:52.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T13:09:52.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=MOBJ0 2026-02-19T13:09:52.618 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool MOBJ0 2026-02-19T13:09:52.618 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:09:52.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-19T13:09:52.780 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:09:52.780 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:09:52.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 1 0 2026-02-19T13:09:52.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:991: list_missing_erasure_coded: osds0=('2' '1' '0') 2026-02-19T13:09:52.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:991: list_missing_erasure_coded: local -a osds0 2026-02-19T13:09:52.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:994: list_missing_erasure_coded: add_something td/osd-scrub-repair ecpool MOBJ1 2026-02-19T13:09:52.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:09:52.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T13:09:52.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=MOBJ1 2026-02-19T13:09:52.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:09:52.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:09:52.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:09:52.984 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:09:52.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:09:53.190 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:09:53.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:09:53.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:09:53.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put MOBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:09:53.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:995: list_missing_erasure_coded: get_osds ecpool MOBJ1 2026-02-19T13:09:53.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T13:09:53.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=MOBJ1 2026-02-19T13:09:53.222 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool MOBJ1 2026-02-19T13:09:53.222 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:09:53.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-19T13:09:53.381 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:09:53.381 INFO:tasks.workunit.client.0.trial144.stderr:0' 2026-02-19T13:09:53.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 1 0 2026-02-19T13:09:53.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:995: list_missing_erasure_coded: osds1=('2' '1' '0') 2026-02-19T13:09:53.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:995: list_missing_erasure_coded: local -a osds1 2026-02-19T13:09:53.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:998: list_missing_erasure_coded: seq 0 2 2026-02-19T13:09:53.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:998: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:09:53.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:999: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:09:53.383 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:53.383 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:53.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:53.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:53.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:53.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:53.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:998: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:09:53.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:999: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:09:53.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:53.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:53.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:53.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:53.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:54.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:54.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:998: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:09:54.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:999: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-19T13:09:54.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:54.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:54.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:54.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:54.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:54.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:54.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1002: list_missing_erasure_coded: id=2 2026-02-19T13:09:54.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1003: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 MOBJ0 remove 2026-02-19T13:09:54.535 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#2:a9030505:::MOBJ0:head# 2026-02-19T13:09:54.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1005: list_missing_erasure_coded: id=1 2026-02-19T13:09:54.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 MOBJ0 remove 2026-02-19T13:09:54.835 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#2:a9030505:::MOBJ0:head# 2026-02-19T13:09:54.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1009: list_missing_erasure_coded: id=1 2026-02-19T13:09:54.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 MOBJ1 remove 2026-02-19T13:09:55.098 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#2:8dd16d55:::MOBJ1:head# 2026-02-19T13:09:55.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1012: list_missing_erasure_coded: id=0 2026-02-19T13:09:55.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1013: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 MOBJ1 remove 2026-02-19T13:09:55.388 INFO:tasks.workunit.client.0.trial144.stdout:remove 2#2:8dd16d55:::MOBJ1:head# 2026-02-19T13:09:55.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1016: list_missing_erasure_coded: seq 0 2 2026-02-19T13:09:55.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1016: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:09:55.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1017: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 0 2026-02-19T13:09:55.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:09:55.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:09:55.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:09:55.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:09:55.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:09:55.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:09:55.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:55.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:55.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:55.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:09:55.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:09:55.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:55.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:09:55.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:09:55.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:09:55.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:55.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:55.482 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:55.482 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:55.482 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:55.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:55.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:55.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:55.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:09:55.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:09:55.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:55.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:55.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:55.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:55.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:09:55.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:09:55.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:09:55.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:09:55.485 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:09:55.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:09:55.485 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:09:55.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:09:55.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:09:55.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:09:55.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:09:55.511 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:55.512+0000 7f285b151a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:55.513 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:55.512+0000 7f285b151a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:55.513 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:55.512+0000 7f285b151a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:55.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:09:55.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:55.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:09:55.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:55.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:55.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:55.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:55.647 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:09:55.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:55.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:55.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:55.933 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:55.932+0000 7f285b151a40 -1 Falling back to public interface 2026-02-19T13:09:56.486 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:56.484+0000 7f285b151a40 -1 osd.0 31 log_to_monitors true 2026-02-19T13:09:56.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:56.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:56.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:56.811 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:09:56.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:56.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:56.967 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 38 up_thru 38 down_at 32 last_clean_interval [5,31) [v2:127.0.0.1:6802/3407520186,v1:127.0.0.1:6803/3407520186] [v2:127.0.0.1:6804/3407520186,v1:127.0.0.1:6805/3407520186] exists,up 7e0847bc-b958-47d0-8d0f-ddbdc4b2b0b0 2026-02-19T13:09:56.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:56.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:56.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:56.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1016: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:09:56.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1017: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 1 2026-02-19T13:09:56.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:09:56.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:09:56.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:09:56.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:09:56.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:09:56.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:09:56.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:56.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:56.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:56.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:09:56.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:09:56.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:56.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:09:56.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:09:56.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:09:56.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:56.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:56.969 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:56.969 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:56.969 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:56.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:56.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:56.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:56.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:09:56.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:09:56.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:56.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:56.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:56.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:56.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:09:56.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:09:56.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:09:56.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:09:56.970 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:09:56.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:09:56.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:09:56.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:09:56.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:09:56.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:09:56.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:09:56.988 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:56.988+0000 7fdaba3c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:56.989 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:56.988+0000 7fdaba3c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:56.990 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:56.988+0000 7fdaba3c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:57.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:09:57.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:57.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:09:57.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:57.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:57.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:57.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:57.133 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:09:57.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:57.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:09:57.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:57.388 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:57.388+0000 7fdaba3c5a40 -1 Falling back to public interface 2026-02-19T13:09:57.941 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:57.940+0000 7fdaba3c5a40 -1 osd.1 33 log_to_monitors true 2026-02-19T13:09:58.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:58.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:58.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:58.294 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:09:58.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:58.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:09:58.451 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 42 up_thru 42 down_at 34 last_clean_interval [9,33) [v2:127.0.0.1:6810/2768005022,v1:127.0.0.1:6811/2768005022] [v2:127.0.0.1:6812/2768005022,v1:127.0.0.1:6813/2768005022] exists,up 48c6f101-a677-4564-a05f-8b44ac088736 2026-02-19T13:09:58.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:58.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:58.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:58.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1016: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-19T13:09:58.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1017: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 2 2026-02-19T13:09:58.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:09:58.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:09:58.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:09:58.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:09:58.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:09:58.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:09:58.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:58.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:58.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:58.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T13:09:58.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T13:09:58.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:58.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:09:58.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:09:58.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:09:58.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:58.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:58.455 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:58.455 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:58.455 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:09:58.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:58.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:09:58.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:58.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:09:58.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:09:58.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:58.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:58.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:58.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:58.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:09:58.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:09:58.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T13:09:58.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:09:58.456 INFO:tasks.workunit.client.0.trial144.stderr:start osd.2 2026-02-19T13:09:58.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:09:58.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-19T13:09:58.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:09:58.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:09:58.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:09:58.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:09:58.472 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:58.472+0000 7f75672cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:58.473 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:58.472+0000 7f75672cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:58.474 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:58.472+0000 7f75672cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:58.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:09:58.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:58.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:09:58.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:58.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:58.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:58.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:58.621 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:09:58.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:58.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:09:58.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:58.904 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:58.904+0000 7f75672cca40 -1 Falling back to public interface 2026-02-19T13:09:59.494 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:09:59.492+0000 7f75672cca40 -1 osd.2 35 log_to_monitors true 2026-02-19T13:09:59.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:59.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:59.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:59.783 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:09:59.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:59.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:09:59.937 INFO:tasks.workunit.client.0.trial144.stderr:osd.2 up in weight 1 up_from 47 up_thru 47 down_at 36 last_clean_interval [13,35) [v2:127.0.0.1:6818/602472171,v1:127.0.0.1:6819/602472171] [v2:127.0.0.1:6820/602472171,v1:127.0.0.1:6821/602472171] exists,up c1c53704-95e8-412b-909a-f305b6a18309 2026-02-19T13:09:59.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:59.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:59.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:59.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1019: list_missing_erasure_coded: create_rbd_pool 2026-02-19T13:09:59.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T13:10:00.131 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' removed 2026-02-19T13:10:00.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T13:10:00.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:10:00.336 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T13:10:00.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:10:01.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T13:10:01.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1020: list_missing_erasure_coded: wait_for_clean 2026-02-19T13:10:01.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:10:01.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:10:01.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:10:01.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:10:01.634 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:10:01.634 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:10:01.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:10:01.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:10:01.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:10:01.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:10:01.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:10:01.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:10:01.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:10:01.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:10:01.753 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:10:01.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:10:01.916 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:10:01.916 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:10:01.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:10:01.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:01.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:10:01.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757251 2026-02-19T13:10:01.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757251 2026-02-19T13:10:01.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-163208757251' 2026-02-19T13:10:01.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:01.993 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:10:02.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626434 2026-02-19T13:10:02.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626434 2026-02-19T13:10:02.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-163208757251 1-180388626434' 2026-02-19T13:10:02.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:02.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:10:02.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462914 2026-02-19T13:10:02.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462914 2026-02-19T13:10:02.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-163208757251 1-180388626434 2-201863462914' 2026-02-19T13:10:02.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:02.139 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-163208757251 2026-02-19T13:10:02.139 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:02.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:10:02.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-163208757251 2026-02-19T13:10:02.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:02.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757251 2026-02-19T13:10:02.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 163208757251' 2026-02-19T13:10:02.143 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 163208757251 2026-02-19T13:10:02.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:10:02.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 163208757251 2026-02-19T13:10:02.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:10:03.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:10:03.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:10:03.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757251 -lt 163208757251 2026-02-19T13:10:03.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:03.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-180388626434 2026-02-19T13:10:03.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:03.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:10:03.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-180388626434 2026-02-19T13:10:03.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:03.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626434 2026-02-19T13:10:03.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 180388626434' 2026-02-19T13:10:03.467 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 180388626434 2026-02-19T13:10:03.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:10:03.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626434 -lt 180388626434 2026-02-19T13:10:03.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:03.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-201863462914 2026-02-19T13:10:03.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:03.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:10:03.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-201863462914 2026-02-19T13:10:03.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:03.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=201863462914 2026-02-19T13:10:03.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 201863462914' 2026-02-19T13:10:03.636 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 201863462914 2026-02-19T13:10:03.636 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:10:03.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 201863462914 -lt 201863462914 2026-02-19T13:10:03.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:10:03.800 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:10:03.800 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:10:03.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:10:03.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:10:03.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:10:03.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:10:03.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:10:03.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:10:03.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:10:03.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:10:04.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:10:04.148 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:10:04.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:10:04.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:10:04.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:10:04.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:10:04.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:10:04.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1023: list_missing_erasure_coded: get_pg ecpool MOBJ0 2026-02-19T13:10:04.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T13:10:04.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=MOBJ0 2026-02-19T13:10:04.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool MOBJ0 2026-02-19T13:10:04.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:10:04.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1023: list_missing_erasure_coded: local pg=2.0 2026-02-19T13:10:04.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1027: list_missing_erasure_coded: repair 2.0 2026-02-19T13:10:04.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=2.0 2026-02-19T13:10:04.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 2.0 2026-02-19T13:10:04.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T13:10:04.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:10:04.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:10:04.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T13:10:04.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T13:09:48.212124+0000 2026-02-19T13:10:04.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 2.0 2026-02-19T13:10:04.811 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 2.0s0 on osd.2 to repair 2026-02-19T13:10:04.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 2.0 2026-02-19T13:09:48.212124+0000 2026-02-19T13:10:04.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=2.0 2026-02-19T13:10:04.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:09:48.212124+0000 2026-02-19T13:10:04.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:10:04.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:10:04.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:10:04.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T13:10:04.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T13:10:04.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:10:04.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:10:04.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T13:10:04.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:09:48.212124+0000 '>' 2026-02-19T13:09:48.212124+0000 2026-02-19T13:10:04.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:10:05.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:10:05.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:10:05.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T13:10:05.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T13:10:05.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:10:05.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:10:05.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T13:10:06.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:09:48.212124+0000 '>' 2026-02-19T13:09:48.212124+0000 2026-02-19T13:10:06.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:10:07.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:10:07.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:10:07.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T13:10:07.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T13:10:07.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:10:07.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:10:07.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T13:10:07.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:09:48.212124+0000 '>' 2026-02-19T13:09:48.212124+0000 2026-02-19T13:10:07.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:10:08.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:10:08.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:10:08.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T13:10:08.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T13:10:08.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:10:08.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:10:08.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T13:10:08.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:09:48.212124+0000 '>' 2026-02-19T13:09:48.212124+0000 2026-02-19T13:10:08.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:10:09.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:10:09.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:10:09.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T13:10:09.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T13:10:09.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:10:09.477 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:10:09.477 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T13:10:09.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:09:48.212124+0000 '>' 2026-02-19T13:09:48.212124+0000 2026-02-19T13:10:09.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:10:10.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:10:10.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:10:10.637 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-19T13:10:10.637 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=2.0 2026-02-19T13:10:10.637 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:10:10.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:10:10.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-19T13:10:10.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:10:05.671191+0000 '>' 2026-02-19T13:09:48.212124+0000 2026-02-19T13:10:10.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:10:10.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: list_missing_erasure_coded: seq 0 120 2026-02-19T13:10:10.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: list_missing_erasure_coded: for i in $(seq 0 120) 2026-02-19T13:10:10.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: list_missing_erasure_coded: '[' 0 -lt 60 ']' 2026-02-19T13:10:10.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1031: list_missing_erasure_coded: ceph pg 2.0 list_unfound 2026-02-19T13:10:10.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1031: list_missing_erasure_coded: egrep 'MOBJ0|MOBJ1' 2026-02-19T13:10:10.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1031: list_missing_erasure_coded: wc -l 2026-02-19T13:10:10.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1031: list_missing_erasure_coded: matches=2 2026-02-19T13:10:10.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1032: list_missing_erasure_coded: '[' 2 -eq 2 ']' 2026-02-19T13:10:10.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1032: list_missing_erasure_coded: break 2026-02-19T13:10:10.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T13:10:10.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:10:10.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:10.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:10:10.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:10.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:10.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:10.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:10.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:10.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:10.983 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:10.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:10.985 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:10.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:10.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:10.987 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:10.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:10.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:10.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:10.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:10.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:10.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:10.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:10:11.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:11.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:11.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:11.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:10:11.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:11.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:11.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T13:10:11.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T13:10:11.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T13:10:11.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T13:10:11.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:10:11.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:11.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:10:11.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:11.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:11.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:11.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:11.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:11.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:11.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:11.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:11.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:11.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:11.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:11.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:11.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:11.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:11.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:11.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:11.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:11.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:11.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:10:11.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:11.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:11.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:11.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:10:11.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:11.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:11.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T13:10:11.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:10:11.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:11.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:11.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T13:10:11.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:10:11.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:10:11.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:10:11.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T13:10:11.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_periodic_scrub_replicated td/osd-scrub-repair 2026-02-19T13:10:11.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5926: TEST_periodic_scrub_replicated: local dir=td/osd-scrub-repair 2026-02-19T13:10:11.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5927: TEST_periodic_scrub_replicated: local poolname=psr_pool 2026-02-19T13:10:11.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5928: TEST_periodic_scrub_replicated: local objname=POBJ 2026-02-19T13:10:11.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5929: TEST_periodic_scrub_replicated: return 0 2026-02-19T13:10:11.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T13:10:11.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:10:11.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:11.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:10:11.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:11.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:11.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:11.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:11.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:11.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:11.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:11.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:11.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:11.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:11.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:11.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:11.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:11.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:11.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:11.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:11.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:11.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:11.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:10:11.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:11.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:11.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:11.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:10:11.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:11.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:11.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T13:10:11.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T13:10:11.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T13:10:11.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T13:10:11.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:10:11.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:11.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:10:11.045 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:11.045 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:11.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:11.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:11.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:11.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:11.048 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:11.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:11.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:11.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:11.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:11.051 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:11.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:11.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:11.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:11.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:11.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:11.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:11.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:10:11.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:11.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:11.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:11.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:10:11.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:11.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:11.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T13:10:11.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:10:11.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:11.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:11.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T13:10:11.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:10:11.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:10:11.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:10:11.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T13:10:11.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_repair_stats td/osd-scrub-repair 2026-02-19T13:10:11.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:713: TEST_repair_stats: local dir=td/osd-scrub-repair 2026-02-19T13:10:11.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:714: TEST_repair_stats: local poolname=testpool 2026-02-19T13:10:11.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:715: TEST_repair_stats: local OSDS=2 2026-02-19T13:10:11.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:716: TEST_repair_stats: local OBJS=30 2026-02-19T13:10:11.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:718: TEST_repair_stats: local REPAIRS=20 2026-02-19T13:10:11.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:721: TEST_repair_stats: run_mon td/osd-scrub-repair a 2026-02-19T13:10:11.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T13:10:11.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:10:11.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:10:11.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:10:11.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T13:10:11.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T13:10:11.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:10:11.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:11.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:11.086 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:11.087 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:11.087 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:11.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:11.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:10:11.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:10:11.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:10:11.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:11.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:11.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:10:11.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:10:11.116 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:11.116 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:11.116 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:11.116 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:11.116 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:11.116 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:11.117 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:10:11.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:11.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T13:10:11.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:10:11.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:11.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:11.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:10:11.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:10:11.175 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:11.175 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:11.175 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:11.176 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:11.176 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:11.176 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:11.176 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:10:11.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:11.177 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T13:10:11.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:722: TEST_repair_stats: run_mgr td/osd-scrub-repair x 2026-02-19T13:10:11.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T13:10:11.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:10:11.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:10:11.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:10:11.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T13:10:11.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:10:11.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:10:11.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:11.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:11.323 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:11.323 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:11.324 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:11.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:11.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:11.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:11.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:724: TEST_repair_stats: local 'ceph_osd_args=--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T13:10:11.344 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:725: TEST_repair_stats: expr 2 - 1 2026-02-19T13:10:11.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:725: TEST_repair_stats: seq 0 1 2026-02-19T13:10:11.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:725: TEST_repair_stats: for id in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:10:11.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:726: TEST_repair_stats: run_osd td/osd-scrub-repair 0 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:10:11.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:10:11.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:11.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:10:11.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:11.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:10:11.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:10:11.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:11.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:11.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:11.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:10:11.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:10:11.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:11.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:11.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:10:11.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:11.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:11.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:11.353 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:11.354 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:11.354 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:11.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:11.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:11.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:11.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:11.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:11.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:10:11.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:10:11.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:11.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:11.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:11.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:11.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:11.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T13:10:11.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:10:11.355 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:11.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f62a325e-d4d3-4699-9f6d-8b8cdf843b31 2026-02-19T13:10:11.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f62a325e-d4d3-4699-9f6d-8b8cdf843b31' 2026-02-19T13:10:11.356 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 f62a325e-d4d3-4699-9f6d-8b8cdf843b31 2026-02-19T13:10:11.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:11.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCzC5dpsv2RFRAAJCoyFhl8/kGB20T7H95tsA== 2026-02-19T13:10:11.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCzC5dpsv2RFRAAJCoyFhl8/kGB20T7H95tsA=="}' 2026-02-19T13:10:11.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f62a325e-d4d3-4699-9f6d-8b8cdf843b31 -i td/osd-scrub-repair/0/new.json 2026-02-19T13:10:11.451 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:10:11.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T13:10:11.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQCzC5dpsv2RFRAAJCoyFhl8/kGB20T7H95tsA== --osd-uuid f62a325e-d4d3-4699-9f6d-8b8cdf843b31 2026-02-19T13:10:11.478 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:11.475+0000 7f35b72f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:11.480 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:11.479+0000 7f35b72f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:11.480 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:11.479+0000 7f35b72f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:11.481 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:11.479+0000 7f35b72f1a40 -1 bdev(0x564631d36c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:11.482 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:11.479+0000 7f35b72f1a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T13:10:12.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T13:10:12.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:12.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:10:12.155 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:10:12.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:12.350 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:10:12.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:10:12.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:10:12.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:12.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:12.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:12.366 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:12.363+0000 7fb7347bda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:12.368 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:12.367+0000 7fb7347bda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:12.369 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:12.367+0000 7fb7347bda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:12.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:10:12.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:12.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:10:12.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:12.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:12.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:12.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:12.518 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:10:12.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:12.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:12.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:12.776 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:12.775+0000 7fb7347bda40 -1 Falling back to public interface 2026-02-19T13:10:13.358 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:13.355+0000 7fb7347bda40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:10:13.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:13.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:13.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:13.684 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:10:13.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:13.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:13.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:14.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:14.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:14.862 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:10:14.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:10:14.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:14.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:15.021 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1603948851,v1:127.0.0.1:6803/1603948851] [v2:127.0.0.1:6804/1603948851,v1:127.0.0.1:6805/1603948851] exists,up f62a325e-d4d3-4699-9f6d-8b8cdf843b31 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:725: TEST_repair_stats: for id in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:726: TEST_repair_stats: run_osd td/osd-scrub-repair 1 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:10:15.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:15.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:15.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:10:15.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:15.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:15.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:15.023 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:15.023 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:15.023 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:15.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:15.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:15.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:15.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:15.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:15.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:10:15.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:10:15.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:15.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:15.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:15.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:15.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:15.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T13:10:15.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:10:15.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:15.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d64187e7-f18a-4892-80d9-1434a67dda15 2026-02-19T13:10:15.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 d64187e7-f18a-4892-80d9-1434a67dda15' 2026-02-19T13:10:15.028 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 d64187e7-f18a-4892-80d9-1434a67dda15 2026-02-19T13:10:15.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:15.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC3C5dpt7F1AhAAM07q2jDiGf98C27IT7fRqA== 2026-02-19T13:10:15.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC3C5dpt7F1AhAAM07q2jDiGf98C27IT7fRqA=="}' 2026-02-19T13:10:15.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d64187e7-f18a-4892-80d9-1434a67dda15 -i td/osd-scrub-repair/1/new.json 2026-02-19T13:10:15.209 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:10:15.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T13:10:15.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQC3C5dpt7F1AhAAM07q2jDiGf98C27IT7fRqA== --osd-uuid d64187e7-f18a-4892-80d9-1434a67dda15 2026-02-19T13:10:15.246 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:15.243+0000 7fa0d48b0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:15.247 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:15.247+0000 7fa0d48b0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:15.248 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:15.247+0000 7fa0d48b0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:15.249 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:15.247+0000 7fa0d48b0a40 -1 bdev(0x562964448c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:15.249 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:15.247+0000 7fa0d48b0a40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T13:10:15.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T13:10:15.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:15.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:10:15.872 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:10:15.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:16.074 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:10:16.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:10:16.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:10:16.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:16.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:16.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:16.100 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:16.099+0000 7f370a1cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:16.102 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:16.099+0000 7f370a1cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:16.103 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:16.103+0000 7f370a1cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:16.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:10:16.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:16.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:10:16.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:16.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:16.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:16.234 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:10:16.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:16.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:16.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:10:16.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:16.489 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:16.487+0000 7f370a1cca40 -1 Falling back to public interface 2026-02-19T13:10:17.030 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:17.027+0000 7f370a1cca40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:10:17.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:17.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:17.394 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:10:17.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:17.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:17.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:10:17.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:18.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:18.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:18.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:10:18.549 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:10:18.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:18.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:10:18.707 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2043073620,v1:127.0.0.1:6811/2043073620] [v2:127.0.0.1:6812/2043073620,v1:127.0.0.1:6813/2043073620] exists,up d64187e7-f18a-4892-80d9-1434a67dda15 2026-02-19T13:10:18.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:18.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:18.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:18.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:729: TEST_repair_stats: create_pool testpool 1 1 2026-02-19T13:10:18.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create testpool 1 1 2026-02-19T13:10:18.912 INFO:tasks.workunit.client.0.trial144.stderr:pool 'testpool' created 2026-02-19T13:10:18.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:10:19.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:730: TEST_repair_stats: ceph osd pool set testpool size 2 2026-02-19T13:10:20.127 INFO:tasks.workunit.client.0.trial144.stderr:set pool 1 size to 2 2026-02-19T13:10:20.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:731: TEST_repair_stats: wait_for_clean 2026-02-19T13:10:20.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:10:20.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:10:20.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:10:20.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:10:20.142 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:10:20.142 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:10:20.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:10:20.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:10:20.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:10:20.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:10:20.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:10:20.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:10:20.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:10:20.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:10:20.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:10:20.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:10:20.423 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:10:20.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:10:20.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:20.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:10:20.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T13:10:20.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T13:10:20.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T13:10:20.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:20.496 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:10:20.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705666 2026-02-19T13:10:20.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705666 2026-02-19T13:10:20.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705666' 2026-02-19T13:10:20.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:20.571 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T13:10:20.571 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:20.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:10:20.573 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T13:10:20.573 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:20.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T13:10:20.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T13:10:20.575 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T13:10:20.575 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:10:20.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T13:10:20.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:10:21.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:10:21.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:10:21.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-19T13:10:21.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:21.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705666 2026-02-19T13:10:21.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:21.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:10:21.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705666 2026-02-19T13:10:21.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:21.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705666 2026-02-19T13:10:21.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705666' 2026-02-19T13:10:21.902 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705666 2026-02-19T13:10:21.903 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:10:22.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705666 -lt 38654705666 2026-02-19T13:10:22.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:10:22.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:10:22.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:10:22.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:10:22.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:10:22.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:10:22.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:10:22.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:10:22.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:10:22.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:10:22.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:10:22.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:10:22.420 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:10:22.420 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:10:22.420 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:10:22.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:10:22.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:10:22.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:10:22.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:734: TEST_repair_stats: local payload=ABCDEF 2026-02-19T13:10:22.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:735: TEST_repair_stats: echo ABCDEF 2026-02-19T13:10:22.609 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: seq 1 30 2026-02-19T13:10:22.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj1 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj2 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj3 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj4 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj5 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj6 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj7 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj8 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj9 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj10 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj11 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj12 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj13 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj14 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj15 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj16 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:22.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:22.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj17 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:23.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:23.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj18 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:23.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:23.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj19 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:23.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:23.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj20 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:23.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:23.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj21 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:23.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:23.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj22 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:23.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:23.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj23 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:23.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:23.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj24 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:23.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:23.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj25 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:23.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:23.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj26 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:23.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:23.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj27 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:23.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:23.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj28 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:23.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:23.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj29 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:23.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-19T13:10:23.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats: rados --pool testpool put obj30 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:10:23.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:743: TEST_repair_stats: get_not_primary testpool obj1 2026-02-19T13:10:23.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=testpool 2026-02-19T13:10:23.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=obj1 2026-02-19T13:10:23.330 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary testpool obj1 2026-02-19T13:10:23.330 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T13:10:23.330 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T13:10:23.331 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool obj1 2026-02-19T13:10:23.331 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:10:23.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T13:10:23.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map testpool obj1 2026-02-19T13:10:23.496 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T13:10:23.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:743: TEST_repair_stats: local other=0 2026-02-19T13:10:23.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:744: TEST_repair_stats: get_pg testpool obj1 2026-02-19T13:10:23.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=testpool 2026-02-19T13:10:23.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=obj1 2026-02-19T13:10:23.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map testpool obj1 2026-02-19T13:10:23.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:10:23.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:744: TEST_repair_stats: local pgid=1.0 2026-02-19T13:10:23.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:745: TEST_repair_stats: get_primary testpool obj1 2026-02-19T13:10:23.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T13:10:23.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T13:10:23.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool obj1 2026-02-19T13:10:23.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:10:23.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:745: TEST_repair_stats: local primary=1 2026-02-19T13:10:23.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:10:23.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:23.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:23.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:23.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:23.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:24.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:24.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:748: TEST_repair_stats: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:10:24.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:24.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:24.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:24.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:24.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:24.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:24.611 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: seq 1 20 2026-02-19T13:10:24.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:24.613 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 1 % 2 2026-02-19T13:10:24.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=1 2026-02-19T13:10:24.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj1 remove 2026-02-19T13:10:24.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:24.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:24.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:10:24.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:24.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:10:24.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj1 remove 2026-02-19T13:10:24.823 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-19T13:10:24.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:24.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 2 % 2 2026-02-19T13:10:24.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=0 2026-02-19T13:10:24.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj2 remove 2026-02-19T13:10:24.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:24.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:24.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:10:24.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:24.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:10:24.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj2 remove 2026-02-19T13:10:25.127 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:104778fc:::obj2:head# 2026-02-19T13:10:25.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:25.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 3 % 2 2026-02-19T13:10:25.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=1 2026-02-19T13:10:25.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj3 remove 2026-02-19T13:10:25.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:25.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:25.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:10:25.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:25.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:10:25.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj3 remove 2026-02-19T13:10:25.402 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:8dd16f86:::obj3:head# 2026-02-19T13:10:25.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:25.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 4 % 2 2026-02-19T13:10:25.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=0 2026-02-19T13:10:25.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj4 remove 2026-02-19T13:10:25.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:25.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:25.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:10:25.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:25.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:10:25.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj4 remove 2026-02-19T13:10:25.658 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:0ee9ae15:::obj4:head# 2026-02-19T13:10:25.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:25.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 5 % 2 2026-02-19T13:10:25.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=1 2026-02-19T13:10:25.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj5 remove 2026-02-19T13:10:25.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:25.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:25.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:10:25.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:25.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:10:25.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj5 remove 2026-02-19T13:10:25.898 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:c52c9666:::obj5:head# 2026-02-19T13:10:25.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:25.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 6 % 2 2026-02-19T13:10:25.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=0 2026-02-19T13:10:25.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj6 remove 2026-02-19T13:10:25.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:25.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:25.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:10:25.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:25.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:10:25.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj6 remove 2026-02-19T13:10:26.215 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:233a42c1:::obj6:head# 2026-02-19T13:10:26.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:26.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 7 % 2 2026-02-19T13:10:26.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=1 2026-02-19T13:10:26.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj7 remove 2026-02-19T13:10:26.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:26.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:26.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:10:26.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:26.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:10:26.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj7 remove 2026-02-19T13:10:26.479 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:9a09113a:::obj7:head# 2026-02-19T13:10:26.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:26.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 8 % 2 2026-02-19T13:10:26.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=0 2026-02-19T13:10:26.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj8 remove 2026-02-19T13:10:26.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:26.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:26.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:10:26.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:26.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:10:26.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj8 remove 2026-02-19T13:10:26.765 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:a557efb1:::obj8:head# 2026-02-19T13:10:26.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:26.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 9 % 2 2026-02-19T13:10:26.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=1 2026-02-19T13:10:26.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj9 remove 2026-02-19T13:10:26.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:26.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:26.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:10:26.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:26.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:10:26.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj9 remove 2026-02-19T13:10:27.038 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:fe351e27:::obj9:head# 2026-02-19T13:10:27.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:27.136 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 10 % 2 2026-02-19T13:10:27.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=0 2026-02-19T13:10:27.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj10 remove 2026-02-19T13:10:27.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:27.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:27.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:10:27.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:27.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:10:27.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj10 remove 2026-02-19T13:10:27.341 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:d1337354:::obj10:head# 2026-02-19T13:10:27.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:27.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 11 % 2 2026-02-19T13:10:27.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=1 2026-02-19T13:10:27.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj11 remove 2026-02-19T13:10:27.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:27.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:27.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:10:27.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:27.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:10:27.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj11 remove 2026-02-19T13:10:27.623 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:eeae2a94:::obj11:head# 2026-02-19T13:10:27.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:27.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 12 % 2 2026-02-19T13:10:27.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=0 2026-02-19T13:10:27.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj12 remove 2026-02-19T13:10:27.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:27.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:27.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:10:27.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:27.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:10:27.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj12 remove 2026-02-19T13:10:27.903 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:af5f95cb:::obj12:head# 2026-02-19T13:10:28.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:28.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 13 % 2 2026-02-19T13:10:28.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=1 2026-02-19T13:10:28.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj13 remove 2026-02-19T13:10:28.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:28.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:28.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:10:28.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:28.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:10:28.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj13 remove 2026-02-19T13:10:28.226 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:a61ad63e:::obj13:head# 2026-02-19T13:10:28.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:28.339 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 14 % 2 2026-02-19T13:10:28.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=0 2026-02-19T13:10:28.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj14 remove 2026-02-19T13:10:28.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:28.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:28.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:10:28.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:28.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:10:28.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj14 remove 2026-02-19T13:10:28.504 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:e0a44829:::obj14:head# 2026-02-19T13:10:28.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:28.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 15 % 2 2026-02-19T13:10:28.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=1 2026-02-19T13:10:28.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj15 remove 2026-02-19T13:10:28.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:28.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:28.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:10:28.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:28.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:10:28.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj15 remove 2026-02-19T13:10:28.802 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:ab946124:::obj15:head# 2026-02-19T13:10:28.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:28.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 16 % 2 2026-02-19T13:10:28.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=0 2026-02-19T13:10:28.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj16 remove 2026-02-19T13:10:28.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:28.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:28.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:10:28.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:28.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:10:28.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj16 remove 2026-02-19T13:10:29.113 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:461f8b5e:::obj16:head# 2026-02-19T13:10:29.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:29.194 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 17 % 2 2026-02-19T13:10:29.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=1 2026-02-19T13:10:29.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj17 remove 2026-02-19T13:10:29.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:29.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:29.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:10:29.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:29.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:10:29.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj17 remove 2026-02-19T13:10:29.386 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:24fc6e92:::obj17:head# 2026-02-19T13:10:29.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:29.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 18 % 2 2026-02-19T13:10:29.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=0 2026-02-19T13:10:29.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj18 remove 2026-02-19T13:10:29.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:29.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:29.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:10:29.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:29.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:10:29.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj18 remove 2026-02-19T13:10:29.658 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:b9836a99:::obj18:head# 2026-02-19T13:10:29.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:29.748 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 19 % 2 2026-02-19T13:10:29.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=1 2026-02-19T13:10:29.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj19 remove 2026-02-19T13:10:29.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:29.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:29.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:10:29.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:29.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:10:29.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj19 remove 2026-02-19T13:10:29.930 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:00b31b6c:::obj19:head# 2026-02-19T13:10:30.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-19T13:10:30.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: expr 20 % 2 2026-02-19T13:10:30.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:752: TEST_repair_stats: OSD=0 2026-02-19T13:10:30.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:753: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj20 remove 2026-02-19T13:10:30.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:10:30.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:10:30.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:10:30.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:10:30.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:10:30.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj20 remove 2026-02-19T13:10:30.222 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:24b5611b:::obj20:head# 2026-02-19T13:10:30.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:755: TEST_repair_stats: activate_osd td/osd-scrub-repair 1 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:10:30.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:10:30.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:10:30.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:10:30.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:10:30.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:10:30.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:10:30.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:30.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:30.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:30.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:10:30.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:10:30.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:30.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:10:30.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:10:30.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:10:30.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:30.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:30.318 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:30.318 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:30.318 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:30.318 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:30.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:30.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:30.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:10:30.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:10:30.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:30.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:30.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:30.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:30.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:10:30.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T13:10:30.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:10:30.320 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:10:30.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:10:30.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:10:30.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:10:30.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:10:30.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:10:30.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:10:30.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:10:30.345 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:30.343+0000 7fd1ee576a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:30.346 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:30.343+0000 7fd1ee576a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:30.347 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:30.347+0000 7fd1ee576a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:30.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:10:30.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:30.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:10:30.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:30.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:30.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:30.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:30.481 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:10:30.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:30.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:10:30.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:30.720 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:30.719+0000 7fd1ee576a40 -1 Falling back to public interface 2026-02-19T13:10:31.271 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:31.271+0000 7fd1ee576a40 -1 osd.1 16 log_to_monitors true 2026-02-19T13:10:31.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:31.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:31.639 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:10:31.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:31.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:31.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:10:31.797 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 19 up_thru 19 down_at 17 last_clean_interval [9,16) [v2:127.0.0.1:6802/2959215724,v1:127.0.0.1:6803/2959215724] [v2:127.0.0.1:6804/2959215724,v1:127.0.0.1:6805/2959215724] exists,up d64187e7-f18a-4892-80d9-1434a67dda15 2026-02-19T13:10:31.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:31.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:31.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:31.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:756: TEST_repair_stats: activate_osd td/osd-scrub-repair 0 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:10:31.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:10:31.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:10:31.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:10:31.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:10:31.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:10:31.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:10:31.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:31.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:31.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:31.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:10:31.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:10:31.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:31.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:10:31.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:10:31.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:10:31.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:31.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:31.802 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:31.802 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:31.802 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:31.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:31.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:31.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:31.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:10:31.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:10:31.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:31.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:31.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:31.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:31.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:10:31.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T13:10:31.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:10:31.804 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:10:31.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:10:31.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:10:31.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:10:31.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:10:31.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:10:31.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:10:31.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:10:31.825 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:31.823+0000 7f87a384da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:31.826 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:31.823+0000 7f87a384da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:31.827 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:31.827+0000 7f87a384da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:31.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:10:31.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:31.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:10:31.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:31.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:31.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:31.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:31.965 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:10:31.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:31.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:32.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:32.260 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:32.259+0000 7f87a384da40 -1 Falling back to public interface 2026-02-19T13:10:32.866 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:32.863+0000 7f87a384da40 -1 osd.0 14 log_to_monitors true 2026-02-19T13:10:33.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:33.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:33.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:33.123 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:10:33.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:33.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:33.283 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 23 up_thru 0 down_at 15 last_clean_interval [5,14) [v2:127.0.0.1:6810/3059165260,v1:127.0.0.1:6811/3059165260] [v2:127.0.0.1:6812/3059165260,v1:127.0.0.1:6813/3059165260] exists,up f62a325e-d4d3-4699-9f6d-8b8cdf843b31 2026-02-19T13:10:33.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:33.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:33.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:33.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:757: TEST_repair_stats: wait_for_clean 2026-02-19T13:10:33.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:10:33.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:10:33.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:10:33.284 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:10:33.285 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:10:33.285 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:10:33.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:10:33.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:10:33.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:10:33.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:10:33.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:10:33.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:10:33.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:10:33.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:10:33.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:10:33.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:10:33.569 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:10:33.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:10:33.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:33.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:10:33.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247810 2026-02-19T13:10:33.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247810 2026-02-19T13:10:33.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247810' 2026-02-19T13:10:33.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:33.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:10:33.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378626 2026-02-19T13:10:33.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378626 2026-02-19T13:10:33.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247810 1-81604378626' 2026-02-19T13:10:33.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:33.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-98784247810 2026-02-19T13:10:33.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:33.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:10:33.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-98784247810 2026-02-19T13:10:33.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:33.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=98784247810 2026-02-19T13:10:33.722 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 98784247810 2026-02-19T13:10:33.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 98784247810' 2026-02-19T13:10:33.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:10:33.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247810 -lt 98784247810 2026-02-19T13:10:33.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:33.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-81604378626 2026-02-19T13:10:33.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:33.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:10:33.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-81604378626 2026-02-19T13:10:33.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:33.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378626 2026-02-19T13:10:33.896 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 81604378626 2026-02-19T13:10:33.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 81604378626' 2026-02-19T13:10:33.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:10:34.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378626 -lt 81604378626 2026-02-19T13:10:34.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:10:34.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:10:34.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:10:34.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:10:34.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:10:34.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:10:34.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:10:34.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:10:34.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:10:34.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:10:34.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:10:34.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:10:34.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:10:34.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:10:34.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:10:34.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:10:34.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:10:34.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:10:34.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:759: TEST_repair_stats: repair 1.0 2026-02-19T13:10:34.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=1.0 2026-02-19T13:10:34.605 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 1.0 2026-02-19T13:10:34.605 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:10:34.605 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:10:34.606 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:10:34.606 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:10:34.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T13:10:18.912045+0000 2026-02-19T13:10:34.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 1.0 2026-02-19T13:10:34.912 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.0 on osd.1 to repair 2026-02-19T13:10:34.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 1.0 2026-02-19T13:10:18.912045+0000 2026-02-19T13:10:34.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T13:10:34.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:10:18.912045+0000 2026-02-19T13:10:34.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:10:34.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:10:34.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:10:34.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:10:34.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:10:34.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:10:34.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:10:34.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:10:35.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:10:18.912045+0000 '>' 2026-02-19T13:10:18.912045+0000 2026-02-19T13:10:35.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:10:36.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:10:36.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:10:36.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:10:36.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:10:36.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:10:36.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:10:36.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:10:36.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:10:18.912045+0000 '>' 2026-02-19T13:10:18.912045+0000 2026-02-19T13:10:36.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:10:37.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:10:37.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:10:37.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:10:37.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:10:37.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:10:37.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:10:37.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:10:37.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:10:18.912045+0000 '>' 2026-02-19T13:10:18.912045+0000 2026-02-19T13:10:37.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:10:38.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:10:38.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:10:38.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:10:38.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:10:38.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:10:38.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:10:38.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:10:38.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:10:18.912045+0000 '>' 2026-02-19T13:10:18.912045+0000 2026-02-19T13:10:38.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:10:39.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:10:39.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:10:39.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:10:39.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:10:39.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:10:39.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:10:39.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:10:39.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:10:18.912045+0000 '>' 2026-02-19T13:10:18.912045+0000 2026-02-19T13:10:39.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:10:40.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:10:40.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:10:40.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:10:40.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:10:40.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:10:40.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:10:40.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:10:40.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:10:18.912045+0000 '>' 2026-02-19T13:10:18.912045+0000 2026-02-19T13:10:40.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:10:41.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:10:41.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:10:41.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:10:41.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:10:41.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:10:41.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:10:41.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:10:42.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:10:36.365642+0000 '>' 2026-02-19T13:10:18.912045+0000 2026-02-19T13:10:42.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:10:42.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats: wait_for_clean 2026-02-19T13:10:42.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:10:42.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:10:42.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:10:42.051 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:10:42.051 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:10:42.052 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:10:42.052 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:10:42.052 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:10:42.052 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:10:42.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:10:42.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:10:42.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:10:42.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:10:42.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:10:42.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:10:42.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:10:42.323 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:10:42.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:10:42.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:42.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:10:42.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247812 2026-02-19T13:10:42.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247812 2026-02-19T13:10:42.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247812' 2026-02-19T13:10:42.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:42.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:10:42.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378629 2026-02-19T13:10:42.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378629 2026-02-19T13:10:42.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247812 1-81604378629' 2026-02-19T13:10:42.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:42.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-98784247812 2026-02-19T13:10:42.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:42.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:10:42.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-98784247812 2026-02-19T13:10:42.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:42.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=98784247812 2026-02-19T13:10:42.472 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 98784247812 2026-02-19T13:10:42.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 98784247812' 2026-02-19T13:10:42.473 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:10:42.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247811 -lt 98784247812 2026-02-19T13:10:42.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:10:43.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:10:43.636 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:10:43.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247811 -lt 98784247812 2026-02-19T13:10:43.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:10:44.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:10:44.800 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:10:44.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247813 -lt 98784247812 2026-02-19T13:10:44.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:44.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-81604378629 2026-02-19T13:10:44.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:44.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:10:44.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-81604378629 2026-02-19T13:10:44.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:44.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378629 2026-02-19T13:10:44.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 81604378629' 2026-02-19T13:10:44.962 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 81604378629 2026-02-19T13:10:44.962 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:10:45.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378629 -lt 81604378629 2026-02-19T13:10:45.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:10:45.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:10:45.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:10:45.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:10:45.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:10:45.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:10:45.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:10:45.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:10:45.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:10:45.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:10:45.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:10:45.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:10:45.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:10:45.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:10:45.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:10:45.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:10:45.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:10:45.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:10:45.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:761: TEST_repair_stats: ceph pg dump pgs 2026-02-19T13:10:45.830 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T13:10:45.830 INFO:tasks.workunit.client.0.trial144.stdout:1.0 30 0 0 0 0 210 0 0 30 0 30 active+clean 2026-02-19T13:10:36.381662+0000 14'30 25:134 [1,0] 1 [1,0] 1 14'30 2026-02-19T13:10:36.365642+0000 14'30 2026-02-19T13:10:36.365642+0000 0 1 periodic scrub scheduled @ 2026-02-20T13:10:36.365642+0000 20 0 2026-02-19T13:10:45.830 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T13:10:45.830 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T13:10:45.831 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:10:45.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:762: TEST_repair_stats: flush_pg_stats 2026-02-19T13:10:45.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:10:45.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:10:45.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:10:45.999 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:10:45.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:10:45.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:46.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:10:46.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247814 2026-02-19T13:10:46.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247814 2026-02-19T13:10:46.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247814' 2026-02-19T13:10:46.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:46.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:10:46.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378630 2026-02-19T13:10:46.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378630 2026-02-19T13:10:46.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247814 1-81604378630' 2026-02-19T13:10:46.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:46.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-98784247814 2026-02-19T13:10:46.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:46.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:10:46.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-98784247814 2026-02-19T13:10:46.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:46.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=98784247814 2026-02-19T13:10:46.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 98784247814' 2026-02-19T13:10:46.149 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 98784247814 2026-02-19T13:10:46.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:10:46.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247813 -lt 98784247814 2026-02-19T13:10:46.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:10:47.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:10:47.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:10:47.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247813 -lt 98784247814 2026-02-19T13:10:47.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:10:48.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:10:48.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:10:48.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247814 -lt 98784247814 2026-02-19T13:10:48.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:48.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-81604378630 2026-02-19T13:10:48.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:48.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:10:48.641 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-81604378630 2026-02-19T13:10:48.641 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:48.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378630 2026-02-19T13:10:48.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 81604378630' 2026-02-19T13:10:48.643 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 81604378630 2026-02-19T13:10:48.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:10:48.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378631 -lt 81604378630 2026-02-19T13:10:48.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:765: TEST_repair_stats: ceph pg 1.0 query 2026-02-19T13:10:48.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:765: TEST_repair_stats: jq .info.stats.stat_sum 2026-02-19T13:10:48.876 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:10:48.876 INFO:tasks.workunit.client.0.trial144.stdout: "num_bytes": 210, 2026-02-19T13:10:48.876 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects": 30, 2026-02-19T13:10:48.877 INFO:tasks.workunit.client.0.trial144.stdout: "num_object_clones": 0, 2026-02-19T13:10:48.877 INFO:tasks.workunit.client.0.trial144.stdout: "num_object_copies": 60, 2026-02-19T13:10:48.877 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T13:10:48.877 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_missing": 0, 2026-02-19T13:10:48.877 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_degraded": 0, 2026-02-19T13:10:48.877 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_misplaced": 0, 2026-02-19T13:10:48.877 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_unfound": 0, 2026-02-19T13:10:48.877 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_dirty": 30, 2026-02-19T13:10:48.877 INFO:tasks.workunit.client.0.trial144.stdout: "num_whiteouts": 0, 2026-02-19T13:10:48.877 INFO:tasks.workunit.client.0.trial144.stdout: "num_read": 0, 2026-02-19T13:10:48.877 INFO:tasks.workunit.client.0.trial144.stdout: "num_read_kb": 0, 2026-02-19T13:10:48.878 INFO:tasks.workunit.client.0.trial144.stdout: "num_write": 30, 2026-02-19T13:10:48.878 INFO:tasks.workunit.client.0.trial144.stdout: "num_write_kb": 30, 2026-02-19T13:10:48.878 INFO:tasks.workunit.client.0.trial144.stdout: "num_scrub_errors": 0, 2026-02-19T13:10:48.878 INFO:tasks.workunit.client.0.trial144.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T13:10:48.878 INFO:tasks.workunit.client.0.trial144.stdout: "num_deep_scrub_errors": 0, 2026-02-19T13:10:48.878 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_recovered": 20, 2026-02-19T13:10:48.878 INFO:tasks.workunit.client.0.trial144.stdout: "num_bytes_recovered": 140, 2026-02-19T13:10:48.878 INFO:tasks.workunit.client.0.trial144.stdout: "num_keys_recovered": 0, 2026-02-19T13:10:48.878 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_omap": 0, 2026-02-19T13:10:48.878 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T13:10:48.878 INFO:tasks.workunit.client.0.trial144.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T13:10:48.878 INFO:tasks.workunit.client.0.trial144.stdout: "num_flush": 0, 2026-02-19T13:10:48.878 INFO:tasks.workunit.client.0.trial144.stdout: "num_flush_kb": 0, 2026-02-19T13:10:48.879 INFO:tasks.workunit.client.0.trial144.stdout: "num_evict": 0, 2026-02-19T13:10:48.879 INFO:tasks.workunit.client.0.trial144.stdout: "num_evict_kb": 0, 2026-02-19T13:10:48.879 INFO:tasks.workunit.client.0.trial144.stdout: "num_promote": 0, 2026-02-19T13:10:48.879 INFO:tasks.workunit.client.0.trial144.stdout: "num_flush_mode_high": 0, 2026-02-19T13:10:48.879 INFO:tasks.workunit.client.0.trial144.stdout: "num_flush_mode_low": 0, 2026-02-19T13:10:48.879 INFO:tasks.workunit.client.0.trial144.stdout: "num_evict_mode_some": 0, 2026-02-19T13:10:48.879 INFO:tasks.workunit.client.0.trial144.stdout: "num_evict_mode_full": 0, 2026-02-19T13:10:48.879 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_pinned": 0, 2026-02-19T13:10:48.879 INFO:tasks.workunit.client.0.trial144.stdout: "num_legacy_snapsets": 0, 2026-02-19T13:10:48.879 INFO:tasks.workunit.client.0.trial144.stdout: "num_large_omap_objects": 0, 2026-02-19T13:10:48.879 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_manifest": 0, 2026-02-19T13:10:48.879 INFO:tasks.workunit.client.0.trial144.stdout: "num_omap_bytes": 0, 2026-02-19T13:10:48.880 INFO:tasks.workunit.client.0.trial144.stdout: "num_omap_keys": 0, 2026-02-19T13:10:48.880 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_repaired": 20 2026-02-19T13:10:48.880 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:10:48.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:766: TEST_repair_stats: ceph pg 1.0 query 2026-02-19T13:10:48.881 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:766: TEST_repair_stats: jq .info.stats.stat_sum.num_objects_repaired 2026-02-19T13:10:48.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:766: TEST_repair_stats: COUNT=20 2026-02-19T13:10:48.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:767: TEST_repair_stats: test 20 = 20 2026-02-19T13:10:48.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:769: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-19T13:10:48.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:769: TEST_repair_stats: jq '.pg_map.osd_stats[] | select(.osd == 1 )' 2026-02-19T13:10:49.104 INFO:tasks.workunit.client.0.trial144.stderr:dumped all 2026-02-19T13:10:49.113 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:10:49.113 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:10:49.113 INFO:tasks.workunit.client.0.trial144.stdout: "up_from": 19, 2026-02-19T13:10:49.113 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 81604378631, 2026-02-19T13:10:49.113 INFO:tasks.workunit.client.0.trial144.stdout: "num_pgs": 1, 2026-02-19T13:10:49.113 INFO:tasks.workunit.client.0.trial144.stdout: "num_osds": 1, 2026-02-19T13:10:49.113 INFO:tasks.workunit.client.0.trial144.stdout: "num_per_pool_osds": 1, 2026-02-19T13:10:49.113 INFO:tasks.workunit.client.0.trial144.stdout: "num_per_pool_omap_osds": 1, 2026-02-19T13:10:49.114 INFO:tasks.workunit.client.0.trial144.stdout: "kb": 104857600, 2026-02-19T13:10:49.114 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used": 28568, 2026-02-19T13:10:49.114 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_data": 328, 2026-02-19T13:10:49.114 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_omap": 7, 2026-02-19T13:10:49.114 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_meta": 28216, 2026-02-19T13:10:49.114 INFO:tasks.workunit.client.0.trial144.stdout: "kb_avail": 104829032, 2026-02-19T13:10:49.114 INFO:tasks.workunit.client.0.trial144.stdout: "statfs": { 2026-02-19T13:10:49.114 INFO:tasks.workunit.client.0.trial144.stdout: "total": 107374182400, 2026-02-19T13:10:49.114 INFO:tasks.workunit.client.0.trial144.stdout: "available": 107344928768, 2026-02-19T13:10:49.114 INFO:tasks.workunit.client.0.trial144.stdout: "internally_reserved": 0, 2026-02-19T13:10:49.115 INFO:tasks.workunit.client.0.trial144.stdout: "allocated": 335872, 2026-02-19T13:10:49.115 INFO:tasks.workunit.client.0.trial144.stdout: "data_stored": 56895, 2026-02-19T13:10:49.115 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed": 0, 2026-02-19T13:10:49.115 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed_allocated": 0, 2026-02-19T13:10:49.115 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed_original": 0, 2026-02-19T13:10:49.115 INFO:tasks.workunit.client.0.trial144.stdout: "omap_allocated": 7657, 2026-02-19T13:10:49.115 INFO:tasks.workunit.client.0.trial144.stdout: "internal_metadata": 28893719 2026-02-19T13:10:49.115 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:10:49.115 INFO:tasks.workunit.client.0.trial144.stdout: "hb_peers": [ 2026-02-19T13:10:49.116 INFO:tasks.workunit.client.0.trial144.stdout: 0 2026-02-19T13:10:49.116 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:10:49.116 INFO:tasks.workunit.client.0.trial144.stdout: "snap_trim_queue_len": 0, 2026-02-19T13:10:49.116 INFO:tasks.workunit.client.0.trial144.stdout: "num_snap_trimming": 0, 2026-02-19T13:10:49.116 INFO:tasks.workunit.client.0.trial144.stdout: "num_shards_repaired": 10, 2026-02-19T13:10:49.116 INFO:tasks.workunit.client.0.trial144.stdout: "op_queue_age_hist": { 2026-02-19T13:10:49.116 INFO:tasks.workunit.client.0.trial144.stdout: "histogram": [], 2026-02-19T13:10:49.116 INFO:tasks.workunit.client.0.trial144.stdout: "upper_bound": 1 2026-02-19T13:10:49.116 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:10:49.116 INFO:tasks.workunit.client.0.trial144.stdout: "perf_stat": { 2026-02-19T13:10:49.116 INFO:tasks.workunit.client.0.trial144.stdout: "commit_latency_ms": 0, 2026-02-19T13:10:49.116 INFO:tasks.workunit.client.0.trial144.stdout: "apply_latency_ms": 0, 2026-02-19T13:10:49.116 INFO:tasks.workunit.client.0.trial144.stdout: "commit_latency_ns": 0, 2026-02-19T13:10:49.117 INFO:tasks.workunit.client.0.trial144.stdout: "apply_latency_ns": 0 2026-02-19T13:10:49.117 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:10:49.117 INFO:tasks.workunit.client.0.trial144.stdout: "alerts": [] 2026-02-19T13:10:49.117 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:10:49.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:770: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-19T13:10:49.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:770: TEST_repair_stats: jq '.pg_map.osd_stats[] | select(.osd == 1 ).num_shards_repaired' 2026-02-19T13:10:49.260 INFO:tasks.workunit.client.0.trial144.stderr:dumped all 2026-02-19T13:10:49.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:770: TEST_repair_stats: COUNT=10 2026-02-19T13:10:49.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:771: TEST_repair_stats: expr 20 / 2 2026-02-19T13:10:49.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:771: TEST_repair_stats: test 10 = 10 2026-02-19T13:10:49.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:773: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-19T13:10:49.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:773: TEST_repair_stats: jq '.pg_map.osd_stats[] | select(.osd == 0 )' 2026-02-19T13:10:49.420 INFO:tasks.workunit.client.0.trial144.stderr:dumped all 2026-02-19T13:10:49.429 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:10:49.429 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:10:49.429 INFO:tasks.workunit.client.0.trial144.stdout: "up_from": 23, 2026-02-19T13:10:49.429 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 98784247814, 2026-02-19T13:10:49.429 INFO:tasks.workunit.client.0.trial144.stdout: "num_pgs": 1, 2026-02-19T13:10:49.429 INFO:tasks.workunit.client.0.trial144.stdout: "num_osds": 1, 2026-02-19T13:10:49.429 INFO:tasks.workunit.client.0.trial144.stdout: "num_per_pool_osds": 1, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "num_per_pool_omap_osds": 1, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "kb": 104857600, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used": 28568, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_data": 328, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_omap": 6, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_meta": 28217, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "kb_avail": 104829032, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "statfs": { 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "total": 107374182400, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "available": 107344928768, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "internally_reserved": 0, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "allocated": 335872, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "data_stored": 56895, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed": 0, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed_allocated": 0, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed_original": 0, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "omap_allocated": 6354, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "internal_metadata": 28895022 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "hb_peers": [ 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "snap_trim_queue_len": 0, 2026-02-19T13:10:49.430 INFO:tasks.workunit.client.0.trial144.stdout: "num_snap_trimming": 0, 2026-02-19T13:10:49.431 INFO:tasks.workunit.client.0.trial144.stdout: "num_shards_repaired": 10, 2026-02-19T13:10:49.431 INFO:tasks.workunit.client.0.trial144.stdout: "op_queue_age_hist": { 2026-02-19T13:10:49.431 INFO:tasks.workunit.client.0.trial144.stdout: "histogram": [], 2026-02-19T13:10:49.431 INFO:tasks.workunit.client.0.trial144.stdout: "upper_bound": 1 2026-02-19T13:10:49.431 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:10:49.431 INFO:tasks.workunit.client.0.trial144.stdout: "perf_stat": { 2026-02-19T13:10:49.431 INFO:tasks.workunit.client.0.trial144.stdout: "commit_latency_ms": 0, 2026-02-19T13:10:49.431 INFO:tasks.workunit.client.0.trial144.stdout: "apply_latency_ms": 0, 2026-02-19T13:10:49.431 INFO:tasks.workunit.client.0.trial144.stdout: "commit_latency_ns": 0, 2026-02-19T13:10:49.431 INFO:tasks.workunit.client.0.trial144.stdout: "apply_latency_ns": 0 2026-02-19T13:10:49.431 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:10:49.431 INFO:tasks.workunit.client.0.trial144.stdout: "alerts": [] 2026-02-19T13:10:49.431 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:10:49.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:774: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-19T13:10:49.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:774: TEST_repair_stats: jq '.pg_map.osd_stats[] | select(.osd == 0 ).num_shards_repaired' 2026-02-19T13:10:49.580 INFO:tasks.workunit.client.0.trial144.stderr:dumped all 2026-02-19T13:10:49.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:774: TEST_repair_stats: COUNT=10 2026-02-19T13:10:49.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:775: TEST_repair_stats: expr 20 / 2 2026-02-19T13:10:49.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:775: TEST_repair_stats: test 10 = 10 2026-02-19T13:10:49.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:777: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-19T13:10:49.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:777: TEST_repair_stats: jq .pg_map.osd_stats_sum 2026-02-19T13:10:49.742 INFO:tasks.workunit.client.0.trial144.stderr:dumped all 2026-02-19T13:10:49.751 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:10:49.752 INFO:tasks.workunit.client.0.trial144.stdout: "up_from": 0, 2026-02-19T13:10:49.752 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 0, 2026-02-19T13:10:49.752 INFO:tasks.workunit.client.0.trial144.stdout: "num_pgs": 2, 2026-02-19T13:10:49.752 INFO:tasks.workunit.client.0.trial144.stdout: "num_osds": 2, 2026-02-19T13:10:49.752 INFO:tasks.workunit.client.0.trial144.stdout: "num_per_pool_osds": 2, 2026-02-19T13:10:49.752 INFO:tasks.workunit.client.0.trial144.stdout: "num_per_pool_omap_osds": 2, 2026-02-19T13:10:49.752 INFO:tasks.workunit.client.0.trial144.stdout: "kb": 209715200, 2026-02-19T13:10:49.752 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used": 57136, 2026-02-19T13:10:49.752 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_data": 656, 2026-02-19T13:10:49.752 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_omap": 13, 2026-02-19T13:10:49.753 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_meta": 56434, 2026-02-19T13:10:49.753 INFO:tasks.workunit.client.0.trial144.stdout: "kb_avail": 209658064, 2026-02-19T13:10:49.753 INFO:tasks.workunit.client.0.trial144.stdout: "statfs": { 2026-02-19T13:10:49.753 INFO:tasks.workunit.client.0.trial144.stdout: "total": 214748364800, 2026-02-19T13:10:49.753 INFO:tasks.workunit.client.0.trial144.stdout: "available": 214689857536, 2026-02-19T13:10:49.753 INFO:tasks.workunit.client.0.trial144.stdout: "internally_reserved": 0, 2026-02-19T13:10:49.753 INFO:tasks.workunit.client.0.trial144.stdout: "allocated": 671744, 2026-02-19T13:10:49.753 INFO:tasks.workunit.client.0.trial144.stdout: "data_stored": 113790, 2026-02-19T13:10:49.753 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed": 0, 2026-02-19T13:10:49.753 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed_allocated": 0, 2026-02-19T13:10:49.753 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed_original": 0, 2026-02-19T13:10:49.753 INFO:tasks.workunit.client.0.trial144.stdout: "omap_allocated": 14011, 2026-02-19T13:10:49.754 INFO:tasks.workunit.client.0.trial144.stdout: "internal_metadata": 57788741 2026-02-19T13:10:49.754 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:10:49.754 INFO:tasks.workunit.client.0.trial144.stdout: "hb_peers": [], 2026-02-19T13:10:49.754 INFO:tasks.workunit.client.0.trial144.stdout: "snap_trim_queue_len": 0, 2026-02-19T13:10:49.754 INFO:tasks.workunit.client.0.trial144.stdout: "num_snap_trimming": 0, 2026-02-19T13:10:49.754 INFO:tasks.workunit.client.0.trial144.stdout: "num_shards_repaired": 20, 2026-02-19T13:10:49.754 INFO:tasks.workunit.client.0.trial144.stdout: "op_queue_age_hist": { 2026-02-19T13:10:49.754 INFO:tasks.workunit.client.0.trial144.stdout: "histogram": [], 2026-02-19T13:10:49.754 INFO:tasks.workunit.client.0.trial144.stdout: "upper_bound": 1 2026-02-19T13:10:49.754 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:10:49.754 INFO:tasks.workunit.client.0.trial144.stdout: "perf_stat": { 2026-02-19T13:10:49.754 INFO:tasks.workunit.client.0.trial144.stdout: "commit_latency_ms": 0, 2026-02-19T13:10:49.754 INFO:tasks.workunit.client.0.trial144.stdout: "apply_latency_ms": 0, 2026-02-19T13:10:49.755 INFO:tasks.workunit.client.0.trial144.stdout: "commit_latency_ns": 0, 2026-02-19T13:10:49.755 INFO:tasks.workunit.client.0.trial144.stdout: "apply_latency_ns": 0 2026-02-19T13:10:49.755 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:10:49.755 INFO:tasks.workunit.client.0.trial144.stdout: "alerts": [], 2026-02-19T13:10:49.755 INFO:tasks.workunit.client.0.trial144.stdout: "network_ping_times": [] 2026-02-19T13:10:49.755 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:10:49.756 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:778: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-19T13:10:49.756 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:778: TEST_repair_stats: jq .pg_map.osd_stats_sum.num_shards_repaired 2026-02-19T13:10:49.904 INFO:tasks.workunit.client.0.trial144.stderr:dumped all 2026-02-19T13:10:49.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:778: TEST_repair_stats: COUNT=20 2026-02-19T13:10:49.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:779: TEST_repair_stats: test 20 = 20 2026-02-19T13:10:49.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T13:10:49.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:10:49.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:49.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:10:49.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:49.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:49.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:49.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:49.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:50.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:50.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:50.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:50.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:50.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:50.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:50.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:50.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:50.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:50.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:50.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:50.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:50.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:50.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:10:50.045 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:50.045 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:50.045 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:50.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:10:50.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:50.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:50.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T13:10:50.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T13:10:50.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T13:10:50.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T13:10:50.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:10:50.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:50.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:10:50.048 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:50.048 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:50.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:50.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:50.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:50.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:50.051 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:50.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:50.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:50.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:50.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:50.055 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:50.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:50.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:50.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:50.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:50.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:50.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:50.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:10:50.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:50.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:50.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:50.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:10:50.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:50.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:50.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T13:10:50.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:10:50.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:50.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:50.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T13:10:50.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:10:50.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:10:50.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:10:50.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T13:10:50.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_repair_stats_ec td/osd-scrub-repair 2026-02-19T13:10:50.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:783: TEST_repair_stats_ec: local dir=td/osd-scrub-repair 2026-02-19T13:10:50.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:784: TEST_repair_stats_ec: local poolname=testpool 2026-02-19T13:10:50.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:785: TEST_repair_stats_ec: local OSDS=3 2026-02-19T13:10:50.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:786: TEST_repair_stats_ec: local OBJS=30 2026-02-19T13:10:50.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: local REPAIRS=26 2026-02-19T13:10:50.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:789: TEST_repair_stats_ec: local allow_overwrites=false 2026-02-19T13:10:50.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:792: TEST_repair_stats_ec: run_mon td/osd-scrub-repair a 2026-02-19T13:10:50.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T13:10:50.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:10:50.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:10:50.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:10:50.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T13:10:50.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T13:10:50.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:10:50.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:50.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:50.091 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:50.091 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:50.091 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:50.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:50.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:10:50.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:10:50.122 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:10:50.122 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:50.122 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:50.122 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:10:50.123 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:10:50.123 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:50.123 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:50.124 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:50.124 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:50.124 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:50.124 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:50.124 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:10:50.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:50.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T13:10:50.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:10:50.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:50.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:50.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:10:50.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:10:50.181 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:50.181 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:50.181 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:50.181 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:50.181 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:50.181 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:50.181 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:10:50.182 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:50.182 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T13:10:50.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:793: TEST_repair_stats_ec: run_mgr td/osd-scrub-repair x 2026-02-19T13:10:50.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T13:10:50.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:10:50.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:10:50.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:10:50.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T13:10:50.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:10:50.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:10:50.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:50.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:50.327 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:50.327 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:50.327 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:50.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:50.328 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:50.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:50.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:795: TEST_repair_stats_ec: local 'ceph_osd_args=--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T13:10:50.348 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:796: TEST_repair_stats_ec: expr 3 - 1 2026-02-19T13:10:50.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:796: TEST_repair_stats_ec: seq 0 2 2026-02-19T13:10:50.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:796: TEST_repair_stats_ec: for id in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:10:50.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:797: TEST_repair_stats_ec: run_osd td/osd-scrub-repair 0 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:10:50.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:10:50.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:50.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:10:50.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:50.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:10:50.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:10:50.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:50.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:50.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:50.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:10:50.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:10:50.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:50.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:50.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:10:50.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:50.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:50.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:50.354 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:50.354 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:50.354 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:50.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:50.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:50.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:50.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:50.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:50.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:10:50.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:10:50.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:50.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:50.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:50.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:50.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:50.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T13:10:50.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:10:50.357 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:50.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=89b41f09-8c88-4bbf-9f16-298495a1fa43 2026-02-19T13:10:50.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 89b41f09-8c88-4bbf-9f16-298495a1fa43' 2026-02-19T13:10:50.357 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 89b41f09-8c88-4bbf-9f16-298495a1fa43 2026-02-19T13:10:50.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:50.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDaC5dpGg/qFRAAQsuaIuYY6m/yoj3XYcXdqg== 2026-02-19T13:10:50.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDaC5dpGg/qFRAAQsuaIuYY6m/yoj3XYcXdqg=="}' 2026-02-19T13:10:50.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 89b41f09-8c88-4bbf-9f16-298495a1fa43 -i td/osd-scrub-repair/0/new.json 2026-02-19T13:10:50.458 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:10:50.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T13:10:50.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDaC5dpGg/qFRAAQsuaIuYY6m/yoj3XYcXdqg== --osd-uuid 89b41f09-8c88-4bbf-9f16-298495a1fa43 2026-02-19T13:10:50.483 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:50.483+0000 7fc42ce3da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:50.485 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:50.483+0000 7fc42ce3da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:50.486 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:50.483+0000 7fc42ce3da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:50.486 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:50.483+0000 7fc42ce3da40 -1 bdev(0x563fd56cec00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:50.486 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:50.483+0000 7fc42ce3da40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T13:10:51.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T13:10:51.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:51.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:10:51.276 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:10:51.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:51.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:10:51.477 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:10:51.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:10:51.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:51.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:51.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:51.502 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:51.499+0000 7fe454798a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:51.504 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:51.503+0000 7fe454798a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:51.505 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:51.503+0000 7fe454798a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:51.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:10:51.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:51.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:10:51.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:51.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:51.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:51.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:51.641 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:10:51.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:51.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:51.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:51.864 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:51.863+0000 7fe454798a40 -1 Falling back to public interface 2026-02-19T13:10:52.427 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:52.423+0000 7fe454798a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:10:52.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:52.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:52.799 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:10:52.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:52.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:52.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:52.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:53.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:53.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:53.961 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:10:53.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:10:53.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:53.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:54.121 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/185841039,v1:127.0.0.1:6803/185841039] [v2:127.0.0.1:6804/185841039,v1:127.0.0.1:6805/185841039] exists,up 89b41f09-8c88-4bbf-9f16-298495a1fa43 2026-02-19T13:10:54.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:54.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:54.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:54.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:796: TEST_repair_stats_ec: for id in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:10:54.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:797: TEST_repair_stats_ec: run_osd td/osd-scrub-repair 1 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:10:54.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:10:54.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:54.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:10:54.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:54.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:10:54.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:10:54.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:54.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:54.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:54.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:10:54.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:10:54.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:54.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:54.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:10:54.126 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:54.126 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:54.126 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:54.126 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:54.126 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:54.127 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:54.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:54.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:54.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:54.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:54.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:54.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:10:54.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:10:54.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:54.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:54.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:54.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:54.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:54.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T13:10:54.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:10:54.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:54.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6a605961-c8fc-468d-9c8c-f8fef8fa51c4 2026-02-19T13:10:54.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 6a605961-c8fc-468d-9c8c-f8fef8fa51c4' 2026-02-19T13:10:54.129 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 6a605961-c8fc-468d-9c8c-f8fef8fa51c4 2026-02-19T13:10:54.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:54.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDeC5dpL61ICBAArCtqOmPmCDvWJ4lYMb5ftg== 2026-02-19T13:10:54.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDeC5dpL61ICBAArCtqOmPmCDvWJ4lYMb5ftg=="}' 2026-02-19T13:10:54.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6a605961-c8fc-468d-9c8c-f8fef8fa51c4 -i td/osd-scrub-repair/1/new.json 2026-02-19T13:10:54.300 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:10:54.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T13:10:54.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDeC5dpL61ICBAArCtqOmPmCDvWJ4lYMb5ftg== --osd-uuid 6a605961-c8fc-468d-9c8c-f8fef8fa51c4 2026-02-19T13:10:54.338 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:54.335+0000 7fc060336a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:54.340 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:54.339+0000 7fc060336a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:54.341 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:54.339+0000 7fc060336a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:54.342 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:54.339+0000 7fc060336a40 -1 bdev(0x555809feac00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:54.342 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:54.339+0000 7fc060336a40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T13:10:54.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T13:10:54.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:54.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:10:54.976 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:10:54.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:55.182 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:10:55.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:10:55.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:10:55.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:55.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:55.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:55.198 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:55.194+0000 7fda8cda7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:55.199 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:55.198+0000 7fda8cda7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:55.200 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:55.198+0000 7fda8cda7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:55.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:10:55.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:55.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:10:55.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:55.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:55.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:55.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:55.343 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:10:55.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:55.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:10:55.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:55.585 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:55.582+0000 7fda8cda7a40 -1 Falling back to public interface 2026-02-19T13:10:56.202 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:56.198+0000 7fda8cda7a40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:10:56.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:56.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:56.500 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:10:56.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:56.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:56.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:10:56.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:57.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:57.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:57.661 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:10:57.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:10:57.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:57.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:10:57.820 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3663707486,v1:127.0.0.1:6811/3663707486] [v2:127.0.0.1:6812/3663707486,v1:127.0.0.1:6813/3663707486] exists,up 6a605961-c8fc-468d-9c8c-f8fef8fa51c4 2026-02-19T13:10:57.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:57.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:57.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:57.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:796: TEST_repair_stats_ec: for id in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:10:57.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:797: TEST_repair_stats_ec: run_osd td/osd-scrub-repair 2 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:10:57.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:10:57.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:57.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:10:57.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:57.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:10:57.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:10:57.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:57.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:57.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:57.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T13:10:57.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T13:10:57.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:57.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:57.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:10:57.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:57.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:57.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:57.826 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:57.826 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:57.826 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:10:57.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:57.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:10:57.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:57.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:57.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:57.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:10:57.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:10:57.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:57.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:57.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:57.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:57.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:57.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T13:10:57.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T13:10:57.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:57.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=85204806-0e77-49b0-a8d7-60d8aca31717 2026-02-19T13:10:57.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 85204806-0e77-49b0-a8d7-60d8aca31717' 2026-02-19T13:10:57.829 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 85204806-0e77-49b0-a8d7-60d8aca31717 2026-02-19T13:10:57.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:57.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDhC5dpxG4AMhAAvD77cGDuDRW0rhNHiONb1w== 2026-02-19T13:10:57.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDhC5dpxG4AMhAAvD77cGDuDRW0rhNHiONb1w=="}' 2026-02-19T13:10:57.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 85204806-0e77-49b0-a8d7-60d8aca31717 -i td/osd-scrub-repair/2/new.json 2026-02-19T13:10:58.004 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:10:58.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T13:10:58.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDhC5dpxG4AMhAAvD77cGDuDRW0rhNHiONb1w== --osd-uuid 85204806-0e77-49b0-a8d7-60d8aca31717 2026-02-19T13:10:58.030 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:58.026+0000 7f1342764a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:58.032 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:58.030+0000 7f1342764a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:58.032 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:58.030+0000 7f1342764a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:58.033 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:58.030+0000 7f1342764a40 -1 bdev(0x56113b32ac00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:58.033 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:58.030+0000 7f1342764a40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T13:10:58.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T13:10:58.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:58.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:10:58.639 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T13:10:58.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:58.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:10:58.831 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T13:10:58.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:10:58.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:58.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:58.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:58.857 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:58.854+0000 7fd414500a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:58.858 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:58.854+0000 7fd414500a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:58.859 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:58.858+0000 7fd414500a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:58.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:10:58.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:58.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:10:58.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:58.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:58.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:58.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:58.992 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:10:58.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:58.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:10:59.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:59.236 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:59.234+0000 7fd414500a40 -1 Falling back to public interface 2026-02-19T13:10:59.896 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:10:59.894+0000 7fd414500a40 -1 osd.2 0 log_to_monitors true 2026-02-19T13:11:00.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:11:00.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:00.153 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:11:00.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:11:00.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:00.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:11:00.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:11:01.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:11:01.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:01.313 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:11:01.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:11:01.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:01.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:11:01.468 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/651221243,v1:127.0.0.1:6819/651221243] [v2:127.0.0.1:6820/651221243,v1:127.0.0.1:6821/651221243] exists,up 85204806-0e77-49b0-a8d7-60d8aca31717 2026-02-19T13:11:01.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:11:01.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:11:01.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:11:01.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:801: TEST_repair_stats_ec: create_ec_pool testpool false k=2 m=1 2026-02-19T13:11:01.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: create_ec_pool: local pool_name=testpool 2026-02-19T13:11:01.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: create_ec_pool: shift 2026-02-19T13:11:01.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: create_ec_pool: local allow_overwrites=false 2026-02-19T13:11:01.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2525: create_ec_pool: shift 2026-02-19T13:11:01.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-19T13:11:01.672 INFO:tasks.workunit.client.0.trial144.stderr:load: isa 2026-02-19T13:11:01.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: create_ec_pool: create_pool testpool 1 1 erasure myprofile 2026-02-19T13:11:01.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create testpool 1 1 erasure myprofile 2026-02-19T13:11:01.977 INFO:tasks.workunit.client.0.trial144.stderr:pool 'testpool' created 2026-02-19T13:11:01.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:11:02.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: create_ec_pool: '[' false = true ']' 2026-02-19T13:11:02.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2535: create_ec_pool: wait_for_clean 2026-02-19T13:11:02.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:11:02.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:11:02.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:11:02.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:11:02.997 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:11:02.997 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:11:02.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:11:02.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:11:02.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:11:03.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:11:03.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:11:03.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:11:03.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:11:03.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:11:03.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:11:03.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:11:03.282 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:11:03.282 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:11:03.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:11:03.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:03.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:11:03.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T13:11:03.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T13:11:03.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T13:11:03.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:03.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:11:03.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672963 2026-02-19T13:11:03.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672963 2026-02-19T13:11:03.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-42949672963' 2026-02-19T13:11:03.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:03.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:11:03.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509442 2026-02-19T13:11:03.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509442 2026-02-19T13:11:03.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-42949672963 2-64424509442' 2026-02-19T13:11:03.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:03.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T13:11:03.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:03.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:11:03.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T13:11:03.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:03.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T13:11:03.503 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T13:11:03.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T13:11:03.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:03.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T13:11:03.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:11:04.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:11:04.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:04.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T13:11:04.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:11:05.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:11:05.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:06.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836483 2026-02-19T13:11:06.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:06.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672963 2026-02-19T13:11:06.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:06.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:11:06.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672963 2026-02-19T13:11:06.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:06.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672963 2026-02-19T13:11:06.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672963' 2026-02-19T13:11:06.012 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672963 2026-02-19T13:11:06.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:11:06.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672963 -lt 42949672963 2026-02-19T13:11:06.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:06.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509442 2026-02-19T13:11:06.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:06.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:11:06.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509442 2026-02-19T13:11:06.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:06.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509442 2026-02-19T13:11:06.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509442' 2026-02-19T13:11:06.178 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 64424509442 2026-02-19T13:11:06.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:11:06.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509442 -lt 64424509442 2026-02-19T13:11:06.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:11:06.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:06.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:06.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:11:06.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:11:06.539 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:11:06.539 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:11:06.539 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:11:06.539 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:11:06.539 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:11:06.539 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:11:06.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:11:06.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:11:06.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:06.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:06.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:11:06.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:11:06.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:11:06.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2536: create_ec_pool: return 0 2026-02-19T13:11:06.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: TEST_repair_stats_ec: local payload=ABCDEF 2026-02-19T13:11:06.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: TEST_repair_stats_ec: echo ABCDEF 2026-02-19T13:11:06.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: seq 1 30 2026-02-19T13:11:06.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:06.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj1 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:06.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:06.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj2 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:06.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:06.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj3 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:06.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:06.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj4 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:06.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:06.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj5 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj6 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj7 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj8 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj9 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj10 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj11 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj12 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj13 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj14 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj15 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj16 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj17 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj18 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj19 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj20 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj21 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj22 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj23 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj24 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj25 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj26 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj27 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj28 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj29 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:806: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-19T13:11:07.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: TEST_repair_stats_ec: rados --pool testpool put obj30 td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:07.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:813: TEST_repair_stats_ec: get_not_primary testpool obj1 2026-02-19T13:11:07.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=testpool 2026-02-19T13:11:07.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=obj1 2026-02-19T13:11:07.662 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary testpool obj1 2026-02-19T13:11:07.662 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T13:11:07.662 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T13:11:07.662 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool obj1 2026-02-19T13:11:07.662 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:11:07.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T13:11:07.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map testpool obj1 2026-02-19T13:11:07.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T13:11:07.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:813: TEST_repair_stats_ec: local other=0 2026-02-19T13:11:07.979 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:814: TEST_repair_stats_ec: get_pg testpool obj1 2026-02-19T13:11:07.979 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=testpool 2026-02-19T13:11:07.979 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=obj1 2026-02-19T13:11:07.979 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map testpool obj1 2026-02-19T13:11:07.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:11:08.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:814: TEST_repair_stats_ec: local pgid=1.0 2026-02-19T13:11:08.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:815: TEST_repair_stats_ec: get_primary testpool obj1 2026-02-19T13:11:08.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=testpool 2026-02-19T13:11:08.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T13:11:08.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map testpool obj1 2026-02-19T13:11:08.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:11:08.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:815: TEST_repair_stats_ec: local primary=1 2026-02-19T13:11:08.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:817: TEST_repair_stats_ec: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:11:08.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:11:08.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:11:08.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:11:08.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:11:08.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:11:08.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:11:08.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:818: TEST_repair_stats_ec: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-19T13:11:08.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:11:08.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:11:08.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:11:08.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:11:08.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:11:08.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:11:08.932 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: seq 1 26 2026-02-19T13:11:08.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:08.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 1 % 2 2026-02-19T13:11:08.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=1 2026-02-19T13:11:08.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj1 remove 2026-02-19T13:11:08.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:08.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:08.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:11:08.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:08.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:11:08.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj1 remove 2026-02-19T13:11:09.119 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#1:ff7b1f36:::obj1:head# 2026-02-19T13:11:09.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:09.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 2 % 2 2026-02-19T13:11:09.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=0 2026-02-19T13:11:09.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj2 remove 2026-02-19T13:11:09.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:09.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:09.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:11:09.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:09.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:11:09.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj2 remove 2026-02-19T13:11:09.447 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#1:104778fc:::obj2:head# 2026-02-19T13:11:09.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:09.540 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 3 % 2 2026-02-19T13:11:09.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=1 2026-02-19T13:11:09.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj3 remove 2026-02-19T13:11:09.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:09.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:09.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:11:09.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:09.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:11:09.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj3 remove 2026-02-19T13:11:09.721 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#1:8dd16f86:::obj3:head# 2026-02-19T13:11:09.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:09.819 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 4 % 2 2026-02-19T13:11:09.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=0 2026-02-19T13:11:09.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj4 remove 2026-02-19T13:11:09.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:09.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:09.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:11:09.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:09.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:11:09.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj4 remove 2026-02-19T13:11:09.978 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#1:0ee9ae15:::obj4:head# 2026-02-19T13:11:10.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:10.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 5 % 2 2026-02-19T13:11:10.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=1 2026-02-19T13:11:10.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj5 remove 2026-02-19T13:11:10.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:10.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:10.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:11:10.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:10.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:11:10.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj5 remove 2026-02-19T13:11:10.255 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#1:c52c9666:::obj5:head# 2026-02-19T13:11:10.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:10.355 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 6 % 2 2026-02-19T13:11:10.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=0 2026-02-19T13:11:10.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj6 remove 2026-02-19T13:11:10.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:10.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:10.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:11:10.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:10.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:11:10.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj6 remove 2026-02-19T13:11:10.539 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#1:233a42c1:::obj6:head# 2026-02-19T13:11:10.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:10.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 7 % 2 2026-02-19T13:11:10.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=1 2026-02-19T13:11:10.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj7 remove 2026-02-19T13:11:10.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:10.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:10.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:11:10.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:10.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:11:10.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj7 remove 2026-02-19T13:11:10.870 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#1:9a09113a:::obj7:head# 2026-02-19T13:11:10.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:10.967 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 8 % 2 2026-02-19T13:11:10.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=0 2026-02-19T13:11:10.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj8 remove 2026-02-19T13:11:10.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:10.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:10.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:11:10.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:10.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:11:10.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj8 remove 2026-02-19T13:11:11.153 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#1:a557efb1:::obj8:head# 2026-02-19T13:11:11.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:11.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 9 % 2 2026-02-19T13:11:11.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=1 2026-02-19T13:11:11.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj9 remove 2026-02-19T13:11:11.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:11.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:11.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:11:11.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:11.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:11:11.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj9 remove 2026-02-19T13:11:11.410 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#1:fe351e27:::obj9:head# 2026-02-19T13:11:11.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:11.496 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 10 % 2 2026-02-19T13:11:11.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=0 2026-02-19T13:11:11.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj10 remove 2026-02-19T13:11:11.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:11.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:11.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:11:11.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:11.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:11:11.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj10 remove 2026-02-19T13:11:11.670 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#1:d1337354:::obj10:head# 2026-02-19T13:11:11.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:11.763 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 11 % 2 2026-02-19T13:11:11.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=1 2026-02-19T13:11:11.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj11 remove 2026-02-19T13:11:11.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:11.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:11.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:11:11.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:11.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:11:11.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj11 remove 2026-02-19T13:11:11.958 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#1:eeae2a94:::obj11:head# 2026-02-19T13:11:12.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:12.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 12 % 2 2026-02-19T13:11:12.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=0 2026-02-19T13:11:12.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj12 remove 2026-02-19T13:11:12.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:12.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:12.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:11:12.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:12.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:11:12.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj12 remove 2026-02-19T13:11:12.238 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#1:af5f95cb:::obj12:head# 2026-02-19T13:11:12.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:12.343 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 13 % 2 2026-02-19T13:11:12.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=1 2026-02-19T13:11:12.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj13 remove 2026-02-19T13:11:12.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:12.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:12.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:11:12.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:12.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:11:12.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj13 remove 2026-02-19T13:11:12.522 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#1:a61ad63e:::obj13:head# 2026-02-19T13:11:12.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:12.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 14 % 2 2026-02-19T13:11:12.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=0 2026-02-19T13:11:12.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj14 remove 2026-02-19T13:11:12.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:12.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:12.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:11:12.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:12.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:11:12.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj14 remove 2026-02-19T13:11:12.793 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#1:e0a44829:::obj14:head# 2026-02-19T13:11:12.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:12.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 15 % 2 2026-02-19T13:11:12.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=1 2026-02-19T13:11:12.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj15 remove 2026-02-19T13:11:12.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:12.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:12.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:11:12.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:12.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:11:12.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj15 remove 2026-02-19T13:11:13.074 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#1:ab946124:::obj15:head# 2026-02-19T13:11:13.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:13.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 16 % 2 2026-02-19T13:11:13.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=0 2026-02-19T13:11:13.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj16 remove 2026-02-19T13:11:13.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:13.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:13.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:11:13.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:13.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:11:13.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj16 remove 2026-02-19T13:11:13.361 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#1:461f8b5e:::obj16:head# 2026-02-19T13:11:13.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:13.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 17 % 2 2026-02-19T13:11:13.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=1 2026-02-19T13:11:13.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj17 remove 2026-02-19T13:11:13.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:13.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:13.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:11:13.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:13.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:11:13.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj17 remove 2026-02-19T13:11:13.646 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#1:24fc6e92:::obj17:head# 2026-02-19T13:11:13.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:13.752 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 18 % 2 2026-02-19T13:11:13.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=0 2026-02-19T13:11:13.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj18 remove 2026-02-19T13:11:13.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:13.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:13.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:11:13.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:13.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:11:13.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj18 remove 2026-02-19T13:11:13.921 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#1:b9836a99:::obj18:head# 2026-02-19T13:11:14.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:14.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 19 % 2 2026-02-19T13:11:14.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=1 2026-02-19T13:11:14.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj19 remove 2026-02-19T13:11:14.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:14.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:14.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:11:14.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:14.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:11:14.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj19 remove 2026-02-19T13:11:14.200 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#1:00b31b6c:::obj19:head# 2026-02-19T13:11:14.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:14.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 20 % 2 2026-02-19T13:11:14.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=0 2026-02-19T13:11:14.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj20 remove 2026-02-19T13:11:14.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:14.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:14.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:11:14.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:14.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:11:14.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj20 remove 2026-02-19T13:11:14.461 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#1:24b5611b:::obj20:head# 2026-02-19T13:11:14.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:14.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 21 % 2 2026-02-19T13:11:14.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=1 2026-02-19T13:11:14.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj21 remove 2026-02-19T13:11:14.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:14.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:14.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:11:14.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:14.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:11:14.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj21 remove 2026-02-19T13:11:14.759 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#1:1243fe1a:::obj21:head# 2026-02-19T13:11:14.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:14.856 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 22 % 2 2026-02-19T13:11:14.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=0 2026-02-19T13:11:14.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj22 remove 2026-02-19T13:11:14.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:14.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:14.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:11:14.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:14.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:11:14.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj22 remove 2026-02-19T13:11:15.026 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#1:c0c49def:::obj22:head# 2026-02-19T13:11:15.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:15.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 23 % 2 2026-02-19T13:11:15.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=1 2026-02-19T13:11:15.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj23 remove 2026-02-19T13:11:15.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:15.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:15.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:11:15.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:15.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:11:15.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj23 remove 2026-02-19T13:11:15.314 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#1:242c40e0:::obj23:head# 2026-02-19T13:11:15.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:15.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 24 % 2 2026-02-19T13:11:15.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=0 2026-02-19T13:11:15.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj24 remove 2026-02-19T13:11:15.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:15.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:15.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:11:15.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:15.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:11:15.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj24 remove 2026-02-19T13:11:15.589 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#1:7eacede5:::obj24:head# 2026-02-19T13:11:15.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:15.671 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 25 % 2 2026-02-19T13:11:15.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=1 2026-02-19T13:11:15.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj25 remove 2026-02-19T13:11:15.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:15.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:15.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:11:15.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:15.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:11:15.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj25 remove 2026-02-19T13:11:15.863 INFO:tasks.workunit.client.0.trial144.stdout:remove 0#1:7109478f:::obj25:head# 2026-02-19T13:11:15.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-19T13:11:15.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: expr 26 % 2 2026-02-19T13:11:15.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:822: TEST_repair_stats_ec: OSD=0 2026-02-19T13:11:15.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:823: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj26 remove 2026-02-19T13:11:15.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:11:15.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:15.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:11:15.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:15.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:11:15.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj26 remove 2026-02-19T13:11:16.147 INFO:tasks.workunit.client.0.trial144.stdout:remove 1#1:f3c2593a:::obj26:head# 2026-02-19T13:11:16.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:825: TEST_repair_stats_ec: activate_osd td/osd-scrub-repair 1 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:11:16.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:11:16.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:11:16.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:11:16.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:11:16.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:11:16.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:11:16.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:11:16.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:11:16.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:11:16.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:11:16.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:11:16.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:11:16.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:11:16.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:11:16.249 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:11:16.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:11:16.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:11:16.250 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:11:16.250 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:11:16.250 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:11:16.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:11:16.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:11:16.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:11:16.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:11:16.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:11:16.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:11:16.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:11:16.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:11:16.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:11:16.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:11:16.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T13:11:16.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:11:16.253 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:11:16.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:11:16.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:11:16.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-19T13:11:16.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:11:16.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:11:16.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:11:16.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:11:16.275 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:16.274+0000 7fcab6644a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:16.277 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:16.274+0000 7fcab6644a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:16.278 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:16.274+0000 7fcab6644a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:16.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:11:16.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:11:16.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:11:16.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:11:16.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:11:16.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:16.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:11:16.410 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:11:16.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:16.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:11:16.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:11:16.669 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:16.666+0000 7fcab6644a40 -1 Falling back to public interface 2026-02-19T13:11:17.319 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:17.314+0000 7fcab6644a40 -1 osd.1 23 log_to_monitors true 2026-02-19T13:11:17.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:11:17.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:17.570 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:11:17.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:11:17.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:17.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:11:17.733 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 27 up_thru 27 down_at 24 last_clean_interval [10,23) [v2:127.0.0.1:6802/3559457144,v1:127.0.0.1:6803/3559457144] [v2:127.0.0.1:6804/3559457144,v1:127.0.0.1:6805/3559457144] exists,up 6a605961-c8fc-468d-9c8c-f8fef8fa51c4 2026-02-19T13:11:17.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:11:17.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:11:17.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:11:17.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:826: TEST_repair_stats_ec: activate_osd td/osd-scrub-repair 0 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:11:17.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:11:17.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:11:17.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:11:17.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:11:17.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:11:17.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:11:17.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:11:17.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:11:17.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:11:17.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:11:17.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:11:17.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:11:17.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:11:17.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:11:17.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:11:17.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:11:17.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:11:17.738 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:11:17.738 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:11:17.739 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:11:17.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:11:17.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:11:17.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:11:17.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:11:17.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:11:17.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:11:17.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:11:17.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:11:17.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:11:17.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:11:17.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-19T13:11:17.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:11:17.740 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:11:17.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:11:17.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-19T13:11:17.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:11:17.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:11:17.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:11:17.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:11:17.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:11:17.761 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:17.758+0000 7fe92bde4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:17.763 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:17.762+0000 7fe92bde4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:17.764 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:17.762+0000 7fe92bde4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:17.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:11:17.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:11:17.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:11:17.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:11:17.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:11:17.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:17.898 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:11:17.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:11:17.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:17.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:11:18.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:11:18.124 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:18.122+0000 7fe92bde4a40 -1 Falling back to public interface 2026-02-19T13:11:18.794 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:18.790+0000 7fe92bde4a40 -1 osd.0 22 log_to_monitors true 2026-02-19T13:11:19.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:11:19.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:19.050 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:11:19.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:11:19.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:19.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:11:19.206 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 30 up_thru 0 down_at 22 last_clean_interval [5,22) [v2:127.0.0.1:6810/1641096166,v1:127.0.0.1:6811/1641096166] [v2:127.0.0.1:6812/1641096166,v1:127.0.0.1:6813/1641096166] exists,up 89b41f09-8c88-4bbf-9f16-298495a1fa43 2026-02-19T13:11:19.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:11:19.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:11:19.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:11:19.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:827: TEST_repair_stats_ec: wait_for_clean 2026-02-19T13:11:19.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:11:19.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:11:19.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:11:19.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:11:19.209 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:11:19.209 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:11:19.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:11:19.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:11:19.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:11:19.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:11:19.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:11:19.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:11:19.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:11:19.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:11:19.323 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:11:19.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:11:19.484 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:11:19.484 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:11:19.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:11:19.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:19.485 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:11:19.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018882 2026-02-19T13:11:19.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018882 2026-02-19T13:11:19.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-128849018882' 2026-02-19T13:11:19.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:19.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:11:19.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116994 2026-02-19T13:11:19.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116994 2026-02-19T13:11:19.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-128849018882 1-115964116994' 2026-02-19T13:11:19.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:19.637 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:11:19.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509446 2026-02-19T13:11:19.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509446 2026-02-19T13:11:19.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-128849018882 1-115964116994 2-64424509446' 2026-02-19T13:11:19.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:19.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-128849018882 2026-02-19T13:11:19.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:19.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:11:19.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-128849018882 2026-02-19T13:11:19.711 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:19.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018882 2026-02-19T13:11:19.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 128849018882' 2026-02-19T13:11:19.712 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 128849018882 2026-02-19T13:11:19.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:19.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 128849018882 2026-02-19T13:11:19.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:11:20.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:11:20.882 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:21.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018882 -lt 128849018882 2026-02-19T13:11:21.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:21.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-115964116994 2026-02-19T13:11:21.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:21.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:11:21.045 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-115964116994 2026-02-19T13:11:21.045 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:21.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964116994 2026-02-19T13:11:21.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 115964116994' 2026-02-19T13:11:21.048 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 115964116994 2026-02-19T13:11:21.048 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:11:21.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116994 -lt 115964116994 2026-02-19T13:11:21.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:21.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509446 2026-02-19T13:11:21.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:21.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:11:21.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509446 2026-02-19T13:11:21.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:21.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509446 2026-02-19T13:11:21.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509446' 2026-02-19T13:11:21.212 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 64424509446 2026-02-19T13:11:21.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:11:21.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509446 -lt 64424509446 2026-02-19T13:11:21.379 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:11:21.379 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:21.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:21.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:11:21.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:11:21.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:11:21.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:11:21.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:11:21.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:11:21.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:11:21.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:11:21.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:11:21.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:11:21.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:21.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:21.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:11:21.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:11:21.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:11:21.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:829: TEST_repair_stats_ec: repair 1.0 2026-02-19T13:11:21.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=1.0 2026-02-19T13:11:21.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 1.0 2026-02-19T13:11:21.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:11:21.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:11:21.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:11:21.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:11:22.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T13:11:01.977861+0000 2026-02-19T13:11:22.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 1.0 2026-02-19T13:11:22.230 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.0s0 on osd.1 to repair 2026-02-19T13:11:22.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 1.0 2026-02-19T13:11:01.977861+0000 2026-02-19T13:11:22.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T13:11:22.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:11:01.977861+0000 2026-02-19T13:11:22.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:11:22.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:11:22.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:11:22.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:11:22.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:11:22.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:11:22.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:11:22.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:11:22.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:11:01.977861+0000 '>' 2026-02-19T13:11:01.977861+0000 2026-02-19T13:11:22.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:11:23.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:11:23.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:11:23.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:11:23.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:11:23.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:11:23.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:11:23.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:11:23.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:11:22.298559+0000 '>' 2026-02-19T13:11:01.977861+0000 2026-02-19T13:11:23.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:11:23.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:830: TEST_repair_stats_ec: wait_for_clean 2026-02-19T13:11:23.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:11:23.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:11:23.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:11:23.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:11:23.564 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:11:23.564 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:11:23.564 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:11:23.564 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:11:23.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:11:23.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:11:23.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:11:23.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:11:23.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:11:23.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:11:23.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:11:23.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:11:23.841 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:11:23.841 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:11:23.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:11:23.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:23.842 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:11:23.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018884 2026-02-19T13:11:23.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018884 2026-02-19T13:11:23.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-128849018884' 2026-02-19T13:11:23.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:23.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:11:23.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116996 2026-02-19T13:11:23.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116996 2026-02-19T13:11:23.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-128849018884 1-115964116996' 2026-02-19T13:11:23.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:23.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:11:24.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509448 2026-02-19T13:11:24.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509448 2026-02-19T13:11:24.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-128849018884 1-115964116996 2-64424509448' 2026-02-19T13:11:24.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:24.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-128849018884 2026-02-19T13:11:24.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:24.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:11:24.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-128849018884 2026-02-19T13:11:24.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:24.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018884 2026-02-19T13:11:24.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 128849018884' 2026-02-19T13:11:24.067 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 128849018884 2026-02-19T13:11:24.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:24.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018882 -lt 128849018884 2026-02-19T13:11:24.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:11:25.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:11:25.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:25.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018884 -lt 128849018884 2026-02-19T13:11:25.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:25.387 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-115964116996 2026-02-19T13:11:25.387 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:25.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:11:25.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-115964116996 2026-02-19T13:11:25.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:25.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964116996 2026-02-19T13:11:25.391 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 115964116996 2026-02-19T13:11:25.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 115964116996' 2026-02-19T13:11:25.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:11:25.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116996 -lt 115964116996 2026-02-19T13:11:25.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:25.556 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509448 2026-02-19T13:11:25.556 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:25.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:11:25.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509448 2026-02-19T13:11:25.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:25.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509448 2026-02-19T13:11:25.561 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 64424509448 2026-02-19T13:11:25.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509448' 2026-02-19T13:11:25.562 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:11:25.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509448 -lt 64424509448 2026-02-19T13:11:25.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:11:25.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:25.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:25.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:11:25.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:11:25.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:11:25.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:11:25.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:11:25.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:11:25.919 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:11:25.919 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:11:26.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:11:26.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:11:26.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:26.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:26.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:11:26.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:11:26.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:11:26.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: TEST_repair_stats_ec: ceph pg dump pgs 2026-02-19T13:11:26.412 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T13:11:26.412 INFO:tasks.workunit.client.0.trial144.stdout:1.0 30 0 0 0 0 210 0 0 30 0 30 active+clean 2026-02-19T13:11:22.326639+0000 21'30 31:137 [1,0,2] 1 [1,0,2] 1 21'30 2026-02-19T13:11:22.298559+0000 21'30 2026-02-19T13:11:22.298559+0000 0 1 periodic scrub scheduled @ 2026-02-20T13:11:22.298559+0000 17 0 2026-02-19T13:11:26.412 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T13:11:26.412 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T13:11:26.413 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:11:26.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: TEST_repair_stats_ec: flush_pg_stats 2026-02-19T13:11:26.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:11:26.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:11:26.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:11:26.585 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:11:26.585 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:11:26.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:11:26.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:26.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:11:26.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018885 2026-02-19T13:11:26.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018885 2026-02-19T13:11:26.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-128849018885' 2026-02-19T13:11:26.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:26.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:11:26.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116997 2026-02-19T13:11:26.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116997 2026-02-19T13:11:26.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-128849018885 1-115964116997' 2026-02-19T13:11:26.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:26.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:11:26.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509450 2026-02-19T13:11:26.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509450 2026-02-19T13:11:26.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-128849018885 1-115964116997 2-64424509450' 2026-02-19T13:11:26.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:26.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-128849018885 2026-02-19T13:11:26.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:26.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:11:26.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-128849018885 2026-02-19T13:11:26.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:26.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=128849018885 2026-02-19T13:11:26.807 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 128849018885 2026-02-19T13:11:26.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 128849018885' 2026-02-19T13:11:26.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:26.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 128849018885 -lt 128849018885 2026-02-19T13:11:26.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:26.979 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-115964116997 2026-02-19T13:11:26.979 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:26.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:11:26.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-115964116997 2026-02-19T13:11:26.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:26.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964116997 2026-02-19T13:11:26.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 115964116997' 2026-02-19T13:11:26.983 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 115964116997 2026-02-19T13:11:26.983 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:11:27.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116997 -lt 115964116997 2026-02-19T13:11:27.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:27.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509450 2026-02-19T13:11:27.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:27.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:11:27.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509450 2026-02-19T13:11:27.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:27.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509450 2026-02-19T13:11:27.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509450' 2026-02-19T13:11:27.148 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 64424509450 2026-02-19T13:11:27.148 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:11:27.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509450 -lt 64424509450 2026-02-19T13:11:27.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:835: TEST_repair_stats_ec: ceph pg 1.0 query 2026-02-19T13:11:27.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:835: TEST_repair_stats_ec: jq .info.stats.stat_sum 2026-02-19T13:11:27.382 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:11:27.383 INFO:tasks.workunit.client.0.trial144.stdout: "num_bytes": 210, 2026-02-19T13:11:27.383 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects": 30, 2026-02-19T13:11:27.383 INFO:tasks.workunit.client.0.trial144.stdout: "num_object_clones": 0, 2026-02-19T13:11:27.383 INFO:tasks.workunit.client.0.trial144.stdout: "num_object_copies": 90, 2026-02-19T13:11:27.383 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T13:11:27.383 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_missing": 0, 2026-02-19T13:11:27.383 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_degraded": 0, 2026-02-19T13:11:27.383 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_misplaced": 0, 2026-02-19T13:11:27.383 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_unfound": 0, 2026-02-19T13:11:27.383 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_dirty": 30, 2026-02-19T13:11:27.384 INFO:tasks.workunit.client.0.trial144.stdout: "num_whiteouts": 0, 2026-02-19T13:11:27.384 INFO:tasks.workunit.client.0.trial144.stdout: "num_read": 0, 2026-02-19T13:11:27.384 INFO:tasks.workunit.client.0.trial144.stdout: "num_read_kb": 0, 2026-02-19T13:11:27.384 INFO:tasks.workunit.client.0.trial144.stdout: "num_write": 30, 2026-02-19T13:11:27.384 INFO:tasks.workunit.client.0.trial144.stdout: "num_write_kb": 30, 2026-02-19T13:11:27.384 INFO:tasks.workunit.client.0.trial144.stdout: "num_scrub_errors": 0, 2026-02-19T13:11:27.384 INFO:tasks.workunit.client.0.trial144.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T13:11:27.384 INFO:tasks.workunit.client.0.trial144.stdout: "num_deep_scrub_errors": 0, 2026-02-19T13:11:27.384 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_recovered": 26, 2026-02-19T13:11:27.384 INFO:tasks.workunit.client.0.trial144.stdout: "num_bytes_recovered": 182, 2026-02-19T13:11:27.384 INFO:tasks.workunit.client.0.trial144.stdout: "num_keys_recovered": 0, 2026-02-19T13:11:27.384 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_omap": 0, 2026-02-19T13:11:27.385 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T13:11:27.385 INFO:tasks.workunit.client.0.trial144.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T13:11:27.385 INFO:tasks.workunit.client.0.trial144.stdout: "num_flush": 0, 2026-02-19T13:11:27.385 INFO:tasks.workunit.client.0.trial144.stdout: "num_flush_kb": 0, 2026-02-19T13:11:27.385 INFO:tasks.workunit.client.0.trial144.stdout: "num_evict": 0, 2026-02-19T13:11:27.385 INFO:tasks.workunit.client.0.trial144.stdout: "num_evict_kb": 0, 2026-02-19T13:11:27.385 INFO:tasks.workunit.client.0.trial144.stdout: "num_promote": 0, 2026-02-19T13:11:27.385 INFO:tasks.workunit.client.0.trial144.stdout: "num_flush_mode_high": 0, 2026-02-19T13:11:27.385 INFO:tasks.workunit.client.0.trial144.stdout: "num_flush_mode_low": 0, 2026-02-19T13:11:27.385 INFO:tasks.workunit.client.0.trial144.stdout: "num_evict_mode_some": 0, 2026-02-19T13:11:27.385 INFO:tasks.workunit.client.0.trial144.stdout: "num_evict_mode_full": 0, 2026-02-19T13:11:27.385 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_pinned": 0, 2026-02-19T13:11:27.386 INFO:tasks.workunit.client.0.trial144.stdout: "num_legacy_snapsets": 0, 2026-02-19T13:11:27.386 INFO:tasks.workunit.client.0.trial144.stdout: "num_large_omap_objects": 0, 2026-02-19T13:11:27.386 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_manifest": 0, 2026-02-19T13:11:27.386 INFO:tasks.workunit.client.0.trial144.stdout: "num_omap_bytes": 0, 2026-02-19T13:11:27.386 INFO:tasks.workunit.client.0.trial144.stdout: "num_omap_keys": 0, 2026-02-19T13:11:27.386 INFO:tasks.workunit.client.0.trial144.stdout: "num_objects_repaired": 26 2026-02-19T13:11:27.386 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:11:27.387 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:836: TEST_repair_stats_ec: ceph pg 1.0 query 2026-02-19T13:11:27.387 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:836: TEST_repair_stats_ec: jq .info.stats.stat_sum.num_objects_repaired 2026-02-19T13:11:27.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:836: TEST_repair_stats_ec: COUNT=26 2026-02-19T13:11:27.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:837: TEST_repair_stats_ec: test 26 = 26 2026-02-19T13:11:27.460 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:839: TEST_repair_stats_ec: expr 3 - 1 2026-02-19T13:11:27.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:839: TEST_repair_stats_ec: seq 0 2 2026-02-19T13:11:27.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:839: TEST_repair_stats_ec: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:11:27.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:840: TEST_repair_stats_ec: '[' 0 = 0 -o 0 = 1 ']' 2026-02-19T13:11:27.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:841: TEST_repair_stats_ec: expr 26 / 2 2026-02-19T13:11:27.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:841: TEST_repair_stats_ec: repair=13 2026-02-19T13:11:27.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:846: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-19T13:11:27.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:846: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 0 )' 2026-02-19T13:11:27.612 INFO:tasks.workunit.client.0.trial144.stderr:dumped all 2026-02-19T13:11:27.621 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:11:27.621 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:11:27.621 INFO:tasks.workunit.client.0.trial144.stdout: "up_from": 30, 2026-02-19T13:11:27.621 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 128849018885, 2026-02-19T13:11:27.621 INFO:tasks.workunit.client.0.trial144.stdout: "num_pgs": 1, 2026-02-19T13:11:27.621 INFO:tasks.workunit.client.0.trial144.stdout: "num_osds": 1, 2026-02-19T13:11:27.621 INFO:tasks.workunit.client.0.trial144.stdout: "num_per_pool_osds": 1, 2026-02-19T13:11:27.621 INFO:tasks.workunit.client.0.trial144.stdout: "num_per_pool_omap_osds": 1, 2026-02-19T13:11:27.622 INFO:tasks.workunit.client.0.trial144.stdout: "kb": 104857600, 2026-02-19T13:11:27.622 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used": 28488, 2026-02-19T13:11:27.622 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_data": 376, 2026-02-19T13:11:27.622 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_omap": 5, 2026-02-19T13:11:27.622 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_meta": 28090, 2026-02-19T13:11:27.622 INFO:tasks.workunit.client.0.trial144.stdout: "kb_avail": 104829112, 2026-02-19T13:11:27.622 INFO:tasks.workunit.client.0.trial144.stdout: "statfs": { 2026-02-19T13:11:27.622 INFO:tasks.workunit.client.0.trial144.stdout: "total": 107374182400, 2026-02-19T13:11:27.622 INFO:tasks.workunit.client.0.trial144.stdout: "available": 107345010688, 2026-02-19T13:11:27.622 INFO:tasks.workunit.client.0.trial144.stdout: "internally_reserved": 0, 2026-02-19T13:11:27.622 INFO:tasks.workunit.client.0.trial144.stdout: "allocated": 385024, 2026-02-19T13:11:27.622 INFO:tasks.workunit.client.0.trial144.stdout: "data_stored": 205045, 2026-02-19T13:11:27.623 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed": 0, 2026-02-19T13:11:27.623 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed_allocated": 0, 2026-02-19T13:11:27.623 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed_original": 0, 2026-02-19T13:11:27.623 INFO:tasks.workunit.client.0.trial144.stdout: "omap_allocated": 5703, 2026-02-19T13:11:27.623 INFO:tasks.workunit.client.0.trial144.stdout: "internal_metadata": 28764601 2026-02-19T13:11:27.623 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:11:27.623 INFO:tasks.workunit.client.0.trial144.stdout: "hb_peers": [ 2026-02-19T13:11:27.623 INFO:tasks.workunit.client.0.trial144.stdout: 1, 2026-02-19T13:11:27.623 INFO:tasks.workunit.client.0.trial144.stdout: 2 2026-02-19T13:11:27.623 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:11:27.623 INFO:tasks.workunit.client.0.trial144.stdout: "snap_trim_queue_len": 0, 2026-02-19T13:11:27.624 INFO:tasks.workunit.client.0.trial144.stdout: "num_snap_trimming": 0, 2026-02-19T13:11:27.624 INFO:tasks.workunit.client.0.trial144.stdout: "num_shards_repaired": 13, 2026-02-19T13:11:27.624 INFO:tasks.workunit.client.0.trial144.stdout: "op_queue_age_hist": { 2026-02-19T13:11:27.624 INFO:tasks.workunit.client.0.trial144.stdout: "histogram": [], 2026-02-19T13:11:27.624 INFO:tasks.workunit.client.0.trial144.stdout: "upper_bound": 1 2026-02-19T13:11:27.624 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:11:27.624 INFO:tasks.workunit.client.0.trial144.stdout: "perf_stat": { 2026-02-19T13:11:27.624 INFO:tasks.workunit.client.0.trial144.stdout: "commit_latency_ms": 0, 2026-02-19T13:11:27.624 INFO:tasks.workunit.client.0.trial144.stdout: "apply_latency_ms": 0, 2026-02-19T13:11:27.624 INFO:tasks.workunit.client.0.trial144.stdout: "commit_latency_ns": 0, 2026-02-19T13:11:27.624 INFO:tasks.workunit.client.0.trial144.stdout: "apply_latency_ns": 0 2026-02-19T13:11:27.625 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:11:27.625 INFO:tasks.workunit.client.0.trial144.stdout: "alerts": [] 2026-02-19T13:11:27.625 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:11:27.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:847: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-19T13:11:27.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:847: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 0 ).num_shards_repaired' 2026-02-19T13:11:27.766 INFO:tasks.workunit.client.0.trial144.stderr:dumped all 2026-02-19T13:11:27.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:847: TEST_repair_stats_ec: COUNT=13 2026-02-19T13:11:27.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:848: TEST_repair_stats_ec: test 13 = 13 2026-02-19T13:11:27.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:839: TEST_repair_stats_ec: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:11:27.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:840: TEST_repair_stats_ec: '[' 1 = 0 -o 1 = 1 ']' 2026-02-19T13:11:27.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:841: TEST_repair_stats_ec: expr 26 / 2 2026-02-19T13:11:27.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:841: TEST_repair_stats_ec: repair=13 2026-02-19T13:11:27.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:846: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-19T13:11:27.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:846: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 1 )' 2026-02-19T13:11:27.925 INFO:tasks.workunit.client.0.trial144.stderr:dumped all 2026-02-19T13:11:27.935 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:11:27.935 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:11:27.935 INFO:tasks.workunit.client.0.trial144.stdout: "up_from": 27, 2026-02-19T13:11:27.935 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 115964116997, 2026-02-19T13:11:27.935 INFO:tasks.workunit.client.0.trial144.stdout: "num_pgs": 1, 2026-02-19T13:11:27.935 INFO:tasks.workunit.client.0.trial144.stdout: "num_osds": 1, 2026-02-19T13:11:27.935 INFO:tasks.workunit.client.0.trial144.stdout: "num_per_pool_osds": 1, 2026-02-19T13:11:27.936 INFO:tasks.workunit.client.0.trial144.stdout: "num_per_pool_omap_osds": 1, 2026-02-19T13:11:27.936 INFO:tasks.workunit.client.0.trial144.stdout: "kb": 104857600, 2026-02-19T13:11:27.936 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used": 28488, 2026-02-19T13:11:27.936 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_data": 376, 2026-02-19T13:11:27.936 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_omap": 6, 2026-02-19T13:11:27.936 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_meta": 28089, 2026-02-19T13:11:27.936 INFO:tasks.workunit.client.0.trial144.stdout: "kb_avail": 104829112, 2026-02-19T13:11:27.936 INFO:tasks.workunit.client.0.trial144.stdout: "statfs": { 2026-02-19T13:11:27.936 INFO:tasks.workunit.client.0.trial144.stdout: "total": 107374182400, 2026-02-19T13:11:27.936 INFO:tasks.workunit.client.0.trial144.stdout: "available": 107345010688, 2026-02-19T13:11:27.936 INFO:tasks.workunit.client.0.trial144.stdout: "internally_reserved": 0, 2026-02-19T13:11:27.936 INFO:tasks.workunit.client.0.trial144.stdout: "allocated": 385024, 2026-02-19T13:11:27.937 INFO:tasks.workunit.client.0.trial144.stdout: "data_stored": 205045, 2026-02-19T13:11:27.937 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed": 0, 2026-02-19T13:11:27.937 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed_allocated": 0, 2026-02-19T13:11:27.937 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed_original": 0, 2026-02-19T13:11:27.937 INFO:tasks.workunit.client.0.trial144.stdout: "omap_allocated": 7003, 2026-02-19T13:11:27.937 INFO:tasks.workunit.client.0.trial144.stdout: "internal_metadata": 28763301 2026-02-19T13:11:27.937 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:11:27.937 INFO:tasks.workunit.client.0.trial144.stdout: "hb_peers": [ 2026-02-19T13:11:27.937 INFO:tasks.workunit.client.0.trial144.stdout: 0, 2026-02-19T13:11:27.937 INFO:tasks.workunit.client.0.trial144.stdout: 2 2026-02-19T13:11:27.937 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:11:27.937 INFO:tasks.workunit.client.0.trial144.stdout: "snap_trim_queue_len": 0, 2026-02-19T13:11:27.937 INFO:tasks.workunit.client.0.trial144.stdout: "num_snap_trimming": 0, 2026-02-19T13:11:27.938 INFO:tasks.workunit.client.0.trial144.stdout: "num_shards_repaired": 13, 2026-02-19T13:11:27.938 INFO:tasks.workunit.client.0.trial144.stdout: "op_queue_age_hist": { 2026-02-19T13:11:27.938 INFO:tasks.workunit.client.0.trial144.stdout: "histogram": [], 2026-02-19T13:11:27.938 INFO:tasks.workunit.client.0.trial144.stdout: "upper_bound": 1 2026-02-19T13:11:27.938 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:11:27.938 INFO:tasks.workunit.client.0.trial144.stdout: "perf_stat": { 2026-02-19T13:11:27.938 INFO:tasks.workunit.client.0.trial144.stdout: "commit_latency_ms": 0, 2026-02-19T13:11:27.938 INFO:tasks.workunit.client.0.trial144.stdout: "apply_latency_ms": 0, 2026-02-19T13:11:27.938 INFO:tasks.workunit.client.0.trial144.stdout: "commit_latency_ns": 0, 2026-02-19T13:11:27.938 INFO:tasks.workunit.client.0.trial144.stdout: "apply_latency_ns": 0 2026-02-19T13:11:27.938 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:11:27.938 INFO:tasks.workunit.client.0.trial144.stdout: "alerts": [] 2026-02-19T13:11:27.939 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:11:27.939 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:847: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-19T13:11:27.939 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:847: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 1 ).num_shards_repaired' 2026-02-19T13:11:28.088 INFO:tasks.workunit.client.0.trial144.stderr:dumped all 2026-02-19T13:11:28.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:847: TEST_repair_stats_ec: COUNT=13 2026-02-19T13:11:28.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:848: TEST_repair_stats_ec: test 13 = 13 2026-02-19T13:11:28.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:839: TEST_repair_stats_ec: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:11:28.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:840: TEST_repair_stats_ec: '[' 2 = 0 -o 2 = 1 ']' 2026-02-19T13:11:28.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:843: TEST_repair_stats_ec: repair=0 2026-02-19T13:11:28.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:846: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-19T13:11:28.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:846: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 2 )' 2026-02-19T13:11:28.245 INFO:tasks.workunit.client.0.trial144.stderr:dumped all 2026-02-19T13:11:28.255 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:11:28.256 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 2, 2026-02-19T13:11:28.256 INFO:tasks.workunit.client.0.trial144.stdout: "up_from": 15, 2026-02-19T13:11:28.256 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 64424509450, 2026-02-19T13:11:28.256 INFO:tasks.workunit.client.0.trial144.stdout: "num_pgs": 1, 2026-02-19T13:11:28.256 INFO:tasks.workunit.client.0.trial144.stdout: "num_osds": 1, 2026-02-19T13:11:28.256 INFO:tasks.workunit.client.0.trial144.stdout: "num_per_pool_osds": 1, 2026-02-19T13:11:28.256 INFO:tasks.workunit.client.0.trial144.stdout: "num_per_pool_omap_osds": 1, 2026-02-19T13:11:28.256 INFO:tasks.workunit.client.0.trial144.stdout: "kb": 104857600, 2026-02-19T13:11:28.256 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used": 27336, 2026-02-19T13:11:28.256 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_data": 376, 2026-02-19T13:11:28.257 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_omap": 9, 2026-02-19T13:11:28.257 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_meta": 26934, 2026-02-19T13:11:28.257 INFO:tasks.workunit.client.0.trial144.stdout: "kb_avail": 104830264, 2026-02-19T13:11:28.257 INFO:tasks.workunit.client.0.trial144.stdout: "statfs": { 2026-02-19T13:11:28.257 INFO:tasks.workunit.client.0.trial144.stdout: "total": 107374182400, 2026-02-19T13:11:28.257 INFO:tasks.workunit.client.0.trial144.stdout: "available": 107346190336, 2026-02-19T13:11:28.257 INFO:tasks.workunit.client.0.trial144.stdout: "internally_reserved": 0, 2026-02-19T13:11:28.257 INFO:tasks.workunit.client.0.trial144.stdout: "allocated": 385024, 2026-02-19T13:11:28.257 INFO:tasks.workunit.client.0.trial144.stdout: "data_stored": 205045, 2026-02-19T13:11:28.257 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed": 0, 2026-02-19T13:11:28.257 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed_allocated": 0, 2026-02-19T13:11:28.257 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed_original": 0, 2026-02-19T13:11:28.258 INFO:tasks.workunit.client.0.trial144.stdout: "omap_allocated": 10088, 2026-02-19T13:11:28.258 INFO:tasks.workunit.client.0.trial144.stdout: "internal_metadata": 27580568 2026-02-19T13:11:28.258 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:11:28.258 INFO:tasks.workunit.client.0.trial144.stdout: "hb_peers": [ 2026-02-19T13:11:28.258 INFO:tasks.workunit.client.0.trial144.stdout: 0, 2026-02-19T13:11:28.258 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:11:28.258 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:11:28.258 INFO:tasks.workunit.client.0.trial144.stdout: "snap_trim_queue_len": 0, 2026-02-19T13:11:28.258 INFO:tasks.workunit.client.0.trial144.stdout: "num_snap_trimming": 0, 2026-02-19T13:11:28.258 INFO:tasks.workunit.client.0.trial144.stdout: "num_shards_repaired": 0, 2026-02-19T13:11:28.258 INFO:tasks.workunit.client.0.trial144.stdout: "op_queue_age_hist": { 2026-02-19T13:11:28.258 INFO:tasks.workunit.client.0.trial144.stdout: "histogram": [], 2026-02-19T13:11:28.258 INFO:tasks.workunit.client.0.trial144.stdout: "upper_bound": 1 2026-02-19T13:11:28.259 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:11:28.259 INFO:tasks.workunit.client.0.trial144.stdout: "perf_stat": { 2026-02-19T13:11:28.259 INFO:tasks.workunit.client.0.trial144.stdout: "commit_latency_ms": 0, 2026-02-19T13:11:28.259 INFO:tasks.workunit.client.0.trial144.stdout: "apply_latency_ms": 0, 2026-02-19T13:11:28.259 INFO:tasks.workunit.client.0.trial144.stdout: "commit_latency_ns": 0, 2026-02-19T13:11:28.259 INFO:tasks.workunit.client.0.trial144.stdout: "apply_latency_ns": 0 2026-02-19T13:11:28.259 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:11:28.259 INFO:tasks.workunit.client.0.trial144.stdout: "alerts": [] 2026-02-19T13:11:28.259 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:11:28.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:847: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-19T13:11:28.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:847: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 2 ).num_shards_repaired' 2026-02-19T13:11:28.407 INFO:tasks.workunit.client.0.trial144.stderr:dumped all 2026-02-19T13:11:28.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:847: TEST_repair_stats_ec: COUNT=0 2026-02-19T13:11:28.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:848: TEST_repair_stats_ec: test 0 = 0 2026-02-19T13:11:28.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:851: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-19T13:11:28.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:851: TEST_repair_stats_ec: jq .pg_map.osd_stats_sum 2026-02-19T13:11:28.567 INFO:tasks.workunit.client.0.trial144.stderr:dumped all 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "up_from": 0, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 0, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "num_pgs": 3, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "num_osds": 3, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "num_per_pool_osds": 3, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "num_per_pool_omap_osds": 3, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "kb": 314572800, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used": 84312, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_data": 1128, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_omap": 22, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "kb_used_meta": 83113, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "kb_avail": 314488488, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "statfs": { 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "total": 322122547200, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "available": 322036211712, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "internally_reserved": 0, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "allocated": 1155072, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "data_stored": 615135, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed": 0, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed_allocated": 0, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "data_compressed_original": 0, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "omap_allocated": 22794, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "internal_metadata": 85108470 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "hb_peers": [], 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "snap_trim_queue_len": 0, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "num_snap_trimming": 0, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "num_shards_repaired": 26, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "op_queue_age_hist": { 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "histogram": [], 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "upper_bound": 1 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:11:28.578 INFO:tasks.workunit.client.0.trial144.stdout: "perf_stat": { 2026-02-19T13:11:28.579 INFO:tasks.workunit.client.0.trial144.stdout: "commit_latency_ms": 0, 2026-02-19T13:11:28.579 INFO:tasks.workunit.client.0.trial144.stdout: "apply_latency_ms": 0, 2026-02-19T13:11:28.579 INFO:tasks.workunit.client.0.trial144.stdout: "commit_latency_ns": 0, 2026-02-19T13:11:28.579 INFO:tasks.workunit.client.0.trial144.stdout: "apply_latency_ns": 0 2026-02-19T13:11:28.579 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:11:28.579 INFO:tasks.workunit.client.0.trial144.stdout: "alerts": [], 2026-02-19T13:11:28.579 INFO:tasks.workunit.client.0.trial144.stdout: "network_ping_times": [] 2026-02-19T13:11:28.579 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:11:28.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:852: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-19T13:11:28.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:852: TEST_repair_stats_ec: jq .pg_map.osd_stats_sum.num_shards_repaired 2026-02-19T13:11:28.729 INFO:tasks.workunit.client.0.trial144.stderr:dumped all 2026-02-19T13:11:28.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:852: TEST_repair_stats_ec: COUNT=26 2026-02-19T13:11:28.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:853: TEST_repair_stats_ec: test 26 = 26 2026-02-19T13:11:28.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T13:11:28.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:11:28.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:11:28.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:11:28.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:11:28.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:11:28.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:11:28.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:11:28.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:11:28.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:11:28.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:11:28.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:11:28.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:11:28.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:11:28.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:11:28.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:11:28.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:11:28.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:11:28.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:11:28.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:11:28.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:11:28.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:11:28.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:11:28.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:11:28.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:11:28.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:11:28.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:11:28.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:11:28.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:11:28.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T13:11:28.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T13:11:28.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T13:11:28.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T13:11:28.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:11:28.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:11:28.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:11:28.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:11:28.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:11:28.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:11:28.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:11:28.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:11:28.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:11:28.877 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:11:28.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:11:28.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:11:28.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:11:28.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:11:28.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:11:28.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:11:28.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:11:28.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:11:28.883 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:11:28.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:11:28.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:11:28.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:11:28.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:11:28.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:11:28.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:11:28.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:11:28.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:11:28.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:11:28.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T13:11:28.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:11:28.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:11:28.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:11:28.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T13:11:28.893 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:11:28.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:11:28.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:11:28.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T13:11:28.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_request_scrub_priority td/osd-scrub-repair 2026-02-19T13:11:28.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6369: TEST_request_scrub_priority: local dir=td/osd-scrub-repair 2026-02-19T13:11:28.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6370: TEST_request_scrub_priority: local poolname=psr_pool 2026-02-19T13:11:28.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6371: TEST_request_scrub_priority: local objname=POBJ 2026-02-19T13:11:28.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6372: TEST_request_scrub_priority: local OBJECTS=64 2026-02-19T13:11:28.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6373: TEST_request_scrub_priority: local PGS=8 2026-02-19T13:11:28.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6375: TEST_request_scrub_priority: run_mon td/osd-scrub-repair a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-19T13:11:28.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T13:11:28.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:11:28.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:11:28.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:11:28.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T13:11:28.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-19T13:11:28.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:11:28.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:11:28.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:11:28.916 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:11:28.917 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:11:28.917 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:11:28.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:11:28.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-19T13:11:28.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:11:28.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:11:28.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:11:28.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:11:28.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:11:28.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:11:28.956 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:11:28.957 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:11:28.957 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:11:28.957 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:11:28.958 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:11:28.958 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:11:28.958 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:11:28.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:11:28.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T13:11:29.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:11:29.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:11:29.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:11:29.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:11:29.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:11:29.015 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:11:29.016 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:11:29.016 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:11:29.016 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:11:29.016 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:11:29.016 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:11:29.016 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:11:29.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:11:29.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T13:11:29.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6376: TEST_request_scrub_priority: run_mgr td/osd-scrub-repair x 2026-02-19T13:11:29.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T13:11:29.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:11:29.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:11:29.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:11:29.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T13:11:29.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:11:29.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:11:29.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:11:29.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:11:29.168 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:11:29.168 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:11:29.168 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:11:29.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:11:29.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:11:29.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:11:29.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6377: TEST_request_scrub_priority: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 ' 2026-02-19T13:11:29.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6378: TEST_request_scrub_priority: ceph_osd_args+=--osd_scrub_backoff_ratio=0 2026-02-19T13:11:29.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6379: TEST_request_scrub_priority: run_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-19T13:11:29.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:11:29.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:11:29.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:11:29.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:11:29.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:11:29.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:11:29.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:11:29.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:11:29.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:11:29.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:11:29.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:11:29.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:11:29.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:11:29.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:11:29.193 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:11:29.194 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:11:29.194 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:11:29.194 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:11:29.194 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:11:29.194 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:11:29.194 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:11:29.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:11:29.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:11:29.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:11:29.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:11:29.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:11:29.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:11:29.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:11:29.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:11:29.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:11:29.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:11:29.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:11:29.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0' 2026-02-19T13:11:29.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:11:29.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:11:29.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a4eb9339-a12f-4e5e-be24-b7901819611e 2026-02-19T13:11:29.196 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 a4eb9339-a12f-4e5e-be24-b7901819611e 2026-02-19T13:11:29.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 a4eb9339-a12f-4e5e-be24-b7901819611e' 2026-02-19T13:11:29.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:11:29.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQABDJdpAQpCDBAABiAPoHD/m7WFAcyHAbYO/A== 2026-02-19T13:11:29.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQABDJdpAQpCDBAABiAPoHD/m7WFAcyHAbYO/A=="}' 2026-02-19T13:11:29.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a4eb9339-a12f-4e5e-be24-b7901819611e -i td/osd-scrub-repair/0/new.json 2026-02-19T13:11:29.293 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:11:29.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T13:11:29.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --mkfs --key AQABDJdpAQpCDBAABiAPoHD/m7WFAcyHAbYO/A== --osd-uuid a4eb9339-a12f-4e5e-be24-b7901819611e 2026-02-19T13:11:29.319 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:29.318+0000 7fe343209a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:29.321 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:29.318+0000 7fe343209a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:29.321 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:29.318+0000 7fe343209a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:29.322 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:29.318+0000 7fe343209a40 -1 bdev(0x55a4dc97cc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:11:29.322 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:29.318+0000 7fe343209a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T13:11:30.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T13:11:30.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:11:30.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:11:30.072 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:11:30.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:11:30.276 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:11:30.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:11:30.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:11:30.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-19T13:11:30.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:11:30.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:11:30.293 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:30.290+0000 7f4767e43a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:30.295 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:30.294+0000 7f4767e43a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:30.296 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:30.294+0000 7f4767e43a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:30.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:11:30.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:11:30.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:11:30.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:11:30.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:11:30.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:30.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:11:30.439 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:11:30.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:30.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:11:30.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:11:30.640 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:30.638+0000 7f4767e43a40 -1 Falling back to public interface 2026-02-19T13:11:31.189 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:11:31.186+0000 7f4767e43a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:11:31.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:11:31.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:31.602 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:11:31.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:11:31.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:31.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:11:31.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:11:32.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:11:32.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:32.762 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:11:32.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:11:32.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:32.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:11:32.919 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1873897092,v1:127.0.0.1:6803/1873897092] [v2:127.0.0.1:6804/1873897092,v1:127.0.0.1:6805/1873897092] exists,up a4eb9339-a12f-4e5e-be24-b7901819611e 2026-02-19T13:11:32.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:11:32.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:11:32.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:11:32.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6381: TEST_request_scrub_priority: create_pool psr_pool 8 8 2026-02-19T13:11:32.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create psr_pool 8 8 2026-02-19T13:11:33.126 INFO:tasks.workunit.client.0.trial144.stderr:pool 'psr_pool' created 2026-02-19T13:11:33.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:11:34.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6382: TEST_request_scrub_priority: wait_for_clean 2026-02-19T13:11:34.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:11:34.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:11:34.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:11:34.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:11:34.145 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:11:34.145 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:11:34.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:11:34.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:11:34.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:11:34.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:11:34.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:11:34.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:11:34.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:11:34.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:11:34.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:11:34.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:11:34.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:11:34.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:34.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:11:34.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836482 2026-02-19T13:11:34.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836482 2026-02-19T13:11:34.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836482' 2026-02-19T13:11:34.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:34.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836482 2026-02-19T13:11:34.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:34.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:11:34.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836482 2026-02-19T13:11:34.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:34.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836482 2026-02-19T13:11:34.508 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836482 2026-02-19T13:11:34.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836482' 2026-02-19T13:11:34.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:34.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836482 2026-02-19T13:11:34.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:11:35.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:11:35.671 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:35.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836482 2026-02-19T13:11:35.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:11:35.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:35.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:36.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 8 == 0 2026-02-19T13:11:36.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:11:36.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:11:36.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:11:36.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:11:36.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:11:36.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:11:36.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:11:36.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=8 2026-02-19T13:11:36.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:11:36.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:36.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:36.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 8 = 8 2026-02-19T13:11:36.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:11:36.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:11:36.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6384: TEST_request_scrub_priority: local osd=0 2026-02-19T13:11:36.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6385: TEST_request_scrub_priority: add_something td/osd-scrub-repair psr_pool POBJ noscrub 2026-02-19T13:11:36.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:11:36.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=psr_pool 2026-02-19T13:11:36.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=POBJ 2026-02-19T13:11:36.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:11:36.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:11:36.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:11:36.587 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:11:36.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:11:36.793 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:11:36.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:11:36.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:11:36.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool psr_pool put POBJ td/osd-scrub-repair/ORIGINAL 2026-02-19T13:11:36.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6386: TEST_request_scrub_priority: get_primary psr_pool POBJ 2026-02-19T13:11:36.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=psr_pool 2026-02-19T13:11:36.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=POBJ 2026-02-19T13:11:36.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:11:36.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map psr_pool POBJ 2026-02-19T13:11:36.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6386: TEST_request_scrub_priority: local primary=0 2026-02-19T13:11:36.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6387: TEST_request_scrub_priority: get_pg psr_pool POBJ 2026-02-19T13:11:36.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=psr_pool 2026-02-19T13:11:36.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=POBJ 2026-02-19T13:11:36.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map psr_pool POBJ 2026-02-19T13:11:36.995 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:11:37.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6387: TEST_request_scrub_priority: local pg=1.1 2026-02-19T13:11:37.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6388: TEST_request_scrub_priority: ceph osd dump 2026-02-19T13:11:37.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6388: TEST_request_scrub_priority: grep '^pool.*['\'']psr_pool['\'']' 2026-02-19T13:11:37.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6388: TEST_request_scrub_priority: awk '{ print $2 }' 2026-02-19T13:11:37.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6388: TEST_request_scrub_priority: poolid=1 2026-02-19T13:11:37.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6390: TEST_request_scrub_priority: local otherpgs 2026-02-19T13:11:37.315 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6391: TEST_request_scrub_priority: expr 8 - 1 2026-02-19T13:11:37.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6391: TEST_request_scrub_priority: seq 0 7 2026-02-19T13:11:37.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6391: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-19T13:11:37.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6393: TEST_request_scrub_priority: opg=1.0 2026-02-19T13:11:37.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6394: TEST_request_scrub_priority: '[' 1.0 = 1.1 ']' 2026-02-19T13:11:37.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6397: TEST_request_scrub_priority: otherpgs='1.0 ' 2026-02-19T13:11:37.318 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6398: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-19T13:11:37.318 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.1 2026-02-19T13:11:37.318 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:11:37.318 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:11:37.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-19T13:11:37.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6398: TEST_request_scrub_priority: local other_last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:11:37.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6400: TEST_request_scrub_priority: ceph tell 1.0 schedule-scrub 1.0 2026-02-19T13:11:37.548 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:11:37.548 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:11:37.548 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:11:37.548 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-17T13:11:37.549245+0000" 2026-02-19T13:11:37.548 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:11:37.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6391: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-19T13:11:37.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6393: TEST_request_scrub_priority: opg=1.1 2026-02-19T13:11:37.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6394: TEST_request_scrub_priority: '[' 1.1 = 1.1 ']' 2026-02-19T13:11:37.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6395: TEST_request_scrub_priority: continue 2026-02-19T13:11:37.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6391: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-19T13:11:37.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6393: TEST_request_scrub_priority: opg=1.2 2026-02-19T13:11:37.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6394: TEST_request_scrub_priority: '[' 1.2 = 1.1 ']' 2026-02-19T13:11:37.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6397: TEST_request_scrub_priority: otherpgs='1.0 1.2 ' 2026-02-19T13:11:37.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6398: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-19T13:11:37.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.1 2026-02-19T13:11:37.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:11:37.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:11:37.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-19T13:11:37.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6398: TEST_request_scrub_priority: local other_last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:11:37.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6400: TEST_request_scrub_priority: ceph tell 1.2 schedule-scrub 1.2 2026-02-19T13:11:37.786 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:11:37.786 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:11:37.786 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:11:37.786 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-17T13:11:37.786877+0000" 2026-02-19T13:11:37.786 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:11:37.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6391: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-19T13:11:37.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6393: TEST_request_scrub_priority: opg=1.3 2026-02-19T13:11:37.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6394: TEST_request_scrub_priority: '[' 1.3 = 1.1 ']' 2026-02-19T13:11:37.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6397: TEST_request_scrub_priority: otherpgs='1.0 1.2 1.3 ' 2026-02-19T13:11:37.794 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6398: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-19T13:11:37.794 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.1 2026-02-19T13:11:37.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:11:37.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:11:37.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-19T13:11:37.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6398: TEST_request_scrub_priority: local other_last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:11:37.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6400: TEST_request_scrub_priority: ceph tell 1.3 schedule-scrub 1.3 2026-02-19T13:11:38.023 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:11:38.023 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:11:38.023 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:11:38.023 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-17T13:11:38.024568+0000" 2026-02-19T13:11:38.023 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:11:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6391: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-19T13:11:38.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6393: TEST_request_scrub_priority: opg=1.4 2026-02-19T13:11:38.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6394: TEST_request_scrub_priority: '[' 1.4 = 1.1 ']' 2026-02-19T13:11:38.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6397: TEST_request_scrub_priority: otherpgs='1.0 1.2 1.3 1.4 ' 2026-02-19T13:11:38.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6398: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-19T13:11:38.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.1 2026-02-19T13:11:38.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:11:38.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:11:38.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-19T13:11:38.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6398: TEST_request_scrub_priority: local other_last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:11:38.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6400: TEST_request_scrub_priority: ceph tell 1.4 schedule-scrub 1.4 2026-02-19T13:11:38.261 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:11:38.261 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:11:38.262 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:11:38.262 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-17T13:11:38.262608+0000" 2026-02-19T13:11:38.262 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:11:38.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6391: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-19T13:11:38.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6393: TEST_request_scrub_priority: opg=1.5 2026-02-19T13:11:38.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6394: TEST_request_scrub_priority: '[' 1.5 = 1.1 ']' 2026-02-19T13:11:38.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6397: TEST_request_scrub_priority: otherpgs='1.0 1.2 1.3 1.4 1.5 ' 2026-02-19T13:11:38.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6398: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-19T13:11:38.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.1 2026-02-19T13:11:38.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:11:38.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:11:38.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-19T13:11:38.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6398: TEST_request_scrub_priority: local other_last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:11:38.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6400: TEST_request_scrub_priority: ceph tell 1.5 schedule-scrub 1.5 2026-02-19T13:11:38.496 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:11:38.496 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:11:38.497 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:11:38.497 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-17T13:11:38.497732+0000" 2026-02-19T13:11:38.497 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:11:38.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6391: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-19T13:11:38.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6393: TEST_request_scrub_priority: opg=1.6 2026-02-19T13:11:38.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6394: TEST_request_scrub_priority: '[' 1.6 = 1.1 ']' 2026-02-19T13:11:38.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6397: TEST_request_scrub_priority: otherpgs='1.0 1.2 1.3 1.4 1.5 1.6 ' 2026-02-19T13:11:38.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6398: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-19T13:11:38.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.1 2026-02-19T13:11:38.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:11:38.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:11:38.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-19T13:11:38.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6398: TEST_request_scrub_priority: local other_last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:11:38.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6400: TEST_request_scrub_priority: ceph tell 1.6 schedule-scrub 1.6 2026-02-19T13:11:38.735 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:11:38.735 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:11:38.735 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:11:38.735 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-17T13:11:38.736116+0000" 2026-02-19T13:11:38.735 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:11:38.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6391: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-19T13:11:38.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6393: TEST_request_scrub_priority: opg=1.7 2026-02-19T13:11:38.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6394: TEST_request_scrub_priority: '[' 1.7 = 1.1 ']' 2026-02-19T13:11:38.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6397: TEST_request_scrub_priority: otherpgs='1.0 1.2 1.3 1.4 1.5 1.6 1.7 ' 2026-02-19T13:11:38.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6398: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-19T13:11:38.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.1 2026-02-19T13:11:38.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:11:38.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:11:38.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-19T13:11:38.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6398: TEST_request_scrub_priority: local other_last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:11:38.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6400: TEST_request_scrub_priority: ceph tell 1.7 schedule-scrub 1.7 2026-02-19T13:11:38.972 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:11:38.972 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:11:38.972 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:11:38.972 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-17T13:11:38.973187+0000" 2026-02-19T13:11:38.972 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:11:38.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6403: TEST_request_scrub_priority: sleep 15 2026-02-19T13:11:53.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6404: TEST_request_scrub_priority: flush_pg_stats 2026-02-19T13:11:53.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:11:53.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:11:54.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:11:54.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:11:54.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:54.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:11:54.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-19T13:11:54.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-19T13:11:54.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-19T13:11:54.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:54.214 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-19T13:11:54.214 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:54.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:11:54.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-19T13:11:54.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:54.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-19T13:11:54.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-19T13:11:54.219 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836487 2026-02-19T13:11:54.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:54.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T13:11:54.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:11:55.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:11:55.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:55.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T13:11:55.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:11:56.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:11:56.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:56.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-19T13:11:56.711 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6407: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-19T13:11:56.711 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.1 2026-02-19T13:11:56.711 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:11:56.711 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:11:56.711 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-19T13:11:56.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6407: TEST_request_scrub_priority: local last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:11:56.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6408: TEST_request_scrub_priority: ceph tell 1.1 scrub 2026-02-19T13:11:56.936 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:11:56.936 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:11:56.936 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:11:56.936 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:11:56.936 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:11:56.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6410: TEST_request_scrub_priority: ceph osd unset noscrub 2026-02-19T13:11:57.150 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is unset 2026-02-19T13:11:57.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6411: TEST_request_scrub_priority: ceph osd unset nodeep-scrub 2026-02-19T13:11:57.356 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is unset 2026-02-19T13:11:57.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6413: TEST_request_scrub_priority: wait_for_scrub 1.1 2026-02-19T13:11:33.126430+0000 2026-02-19T13:11:57.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.1 2026-02-19T13:11:57.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:11:57.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:11:57.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:11:57.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:11:57.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.1 last_scrub_stamp 2026-02-19T13:11:57.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.1 2026-02-19T13:11:57.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:11:57.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:11:57.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-19T13:11:57.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:11:57.248781+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:11:57.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:11:57.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6415: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-19T13:11:57.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6417: TEST_request_scrub_priority: wait_for_scrub 1.0 2026-02-19T13:11:33.126430+0000 2026-02-19T13:11:57.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T13:11:57.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:11:57.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:11:57.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:11:57.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:11:57.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:11:57.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:11:57.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:11:57.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:11:57.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:11:57.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:11:57.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:11:58.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:11:58.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:11:58.690 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:11:58.691 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:11:58.691 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:11:58.691 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:11:58.691 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:11:58.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:11:58.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:11:59.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:11:59.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:11:59.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:11:59.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:11:59.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:11:59.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:11:59.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:00.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:00.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:01.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:01.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:01.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:01.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:01.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:01.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:01.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:01.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:01.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:02.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:02.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:02.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:02.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:02.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:02.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:02.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:02.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:02.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:03.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:03.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:03.341 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:03.341 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:03.341 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:03.342 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:03.342 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:03.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:03.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:04.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:04.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:04.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:04.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:04.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:04.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:04.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:04.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:04.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:05.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:05.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:05.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:05.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:05.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:05.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:05.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:05.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:05.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:06.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:06.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:06.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:06.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:06.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:06.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:06.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:06.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:06.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:07.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:07.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:07.979 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:07.979 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:07.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:07.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:07.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:08.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:08.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:09.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:09.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:09.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:09.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:09.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:09.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:09.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:09.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:09.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:10.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:10.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:10.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:10.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:10.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:10.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:10.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:10.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:10.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:11.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:11.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:11.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:11.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:11.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:11.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:11.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:11.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:11.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:12.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:12.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:12.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:12.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:12.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:12.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:12.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:12.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:12.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:13.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:13.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:13.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:13.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:13.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:13.785 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:13.785 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:13.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:13.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:14.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:14.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:14.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:14.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:14.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:14.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:14.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:15.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:15.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:16.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:16.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:16.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:16.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:16.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:16.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:16.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:16.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:16.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:17.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:17.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:17.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:17.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:17.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:17.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:17.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:17.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:17.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:18.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:18.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:18.437 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:18.437 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:18.437 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:18.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:18.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:18.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:18.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:19.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:19.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:19.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:19.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:19.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:19.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:19.605 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:19.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:19.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:20.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:20.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:20.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:20.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:20.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:20.768 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:20.768 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:20.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:20.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:21.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:21.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:21.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:21.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:21.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:21.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:21.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:22.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:22.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:23.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:23.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:23.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:23.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:23.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:23.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:23.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:23.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:23.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:24.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:24.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:24.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:24.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:24.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:24.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:24.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:24.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:24.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:25.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:25.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:25.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:25.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:25.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:25.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:25.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:25.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:25.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:26.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:26.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:26.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:26.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:26.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:26.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:26.577 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:26.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:26.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:27.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:27.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:27.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:27.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:27.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:27.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:27.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:27.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:27.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:28.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:28.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:28.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:28.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:28.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:28.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:28.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:29.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:29.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:30.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:30.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:30.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:30.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:30.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:30.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:30.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:30.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:30.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:31.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:31.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:31.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:31.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:31.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:31.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:31.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:31.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:31.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:32.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:32.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:32.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:32.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:32.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:32.387 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:32.387 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:32.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:32.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:33.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:33.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:33.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:33.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:33.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:33.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:33.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:33.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:33.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:34.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:34.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:34.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:34.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:34.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:34.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:34.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:34.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:34.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:35.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:35.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:35.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:35.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:35.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:35.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:35.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:36.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:36.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:37.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:37.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:37.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:37.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:37.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:37.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:37.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:37.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:37.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:38.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:38.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:38.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:38.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:38.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:38.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:38.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:38.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:38.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:39.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:39.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:39.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:39.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:39.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:39.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:39.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:39.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:39.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:40.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:40.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:40.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:40.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:40.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:40.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:40.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:40.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:40.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:41.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:41.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:41.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:41.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:41.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:41.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:41.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:41.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:41.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:42.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:42.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:42.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:42.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:42.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:42.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:42.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:42.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:37.549245+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:42.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:43.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:43.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:43.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:12:43.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:12:43.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:43.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:43.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:12:44.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:12:38.346029+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:44.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:12:44.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6415: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-19T13:12:44.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6417: TEST_request_scrub_priority: wait_for_scrub 1.2 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:44.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.2 2026-02-19T13:12:44.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:44.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:12:44.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:12:44.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:44.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.2 last_scrub_stamp 2026-02-19T13:12:44.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.2 2026-02-19T13:12:44.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:44.155 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:44.155 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.2") | .last_scrub_stamp' 2026-02-19T13:12:44.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:12:39.377779+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:44.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:12:44.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6415: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-19T13:12:44.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6417: TEST_request_scrub_priority: wait_for_scrub 1.3 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:44.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.3 2026-02-19T13:12:44.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:44.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:12:44.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:12:44.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:44.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-19T13:12:44.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T13:12:44.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:44.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:44.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T13:12:44.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:12:40.347862+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:44.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:12:44.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6415: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-19T13:12:44.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6417: TEST_request_scrub_priority: wait_for_scrub 1.4 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:44.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.4 2026-02-19T13:12:44.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:44.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:12:44.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:12:44.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:44.473 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.4 last_scrub_stamp 2026-02-19T13:12:44.473 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.4 2026-02-19T13:12:44.473 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:44.473 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:44.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.4") | .last_scrub_stamp' 2026-02-19T13:12:44.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:12:41.317178+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:44.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:12:44.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6415: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-19T13:12:44.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6417: TEST_request_scrub_priority: wait_for_scrub 1.5 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:44.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.5 2026-02-19T13:12:44.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:44.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:12:44.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:12:44.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:44.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.5 last_scrub_stamp 2026-02-19T13:12:44.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.5 2026-02-19T13:12:44.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:44.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:44.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.5") | .last_scrub_stamp' 2026-02-19T13:12:44.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:38.497732+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:44.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:45.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:45.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:45.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.5 last_scrub_stamp 2026-02-19T13:12:45.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.5 2026-02-19T13:12:45.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:45.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:45.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.5") | .last_scrub_stamp' 2026-02-19T13:12:45.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:38.497732+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:45.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:46.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:46.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:46.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.5 last_scrub_stamp 2026-02-19T13:12:46.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.5 2026-02-19T13:12:46.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:46.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:46.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.5") | .last_scrub_stamp' 2026-02-19T13:12:47.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:11:38.497732+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:47.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:12:48.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:12:48.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:48.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.5 last_scrub_stamp 2026-02-19T13:12:48.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.5 2026-02-19T13:12:48.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:48.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:48.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.5") | .last_scrub_stamp' 2026-02-19T13:12:48.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:12:43.325397+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:48.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:12:48.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6415: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-19T13:12:48.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6417: TEST_request_scrub_priority: wait_for_scrub 1.6 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:48.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.6 2026-02-19T13:12:48.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:48.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:12:48.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:12:48.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:48.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.6 last_scrub_stamp 2026-02-19T13:12:48.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.6 2026-02-19T13:12:48.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:48.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:48.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.6") | .last_scrub_stamp' 2026-02-19T13:12:48.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:12:44.297499+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:48.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:12:48.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6415: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-19T13:12:48.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6417: TEST_request_scrub_priority: wait_for_scrub 1.7 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:48.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.7 2026-02-19T13:12:48.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:48.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:12:48.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:12:48.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:48.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.7 last_scrub_stamp 2026-02-19T13:12:48.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.7 2026-02-19T13:12:48.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:48.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:48.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.7") | .last_scrub_stamp' 2026-02-19T13:12:48.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:12:45.343536+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:48.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:12:48.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6415: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-19T13:12:48.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6417: TEST_request_scrub_priority: wait_for_scrub 1.1 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:48.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.1 2026-02-19T13:12:48.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:48.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:12:48.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:12:48.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:12:48.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.1 last_scrub_stamp 2026-02-19T13:12:48.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.1 2026-02-19T13:12:48.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:12:48.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:12:48.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-19T13:12:48.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:11:57.248781+0000 '>' 2026-02-19T13:11:33.126430+0000 2026-02-19T13:12:48.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:12:48.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6421: TEST_request_scrub_priority: grep 'log_channel.*scrub ok' td/osd-scrub-repair/osd.0.log 2026-02-19T13:12:48.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6421: TEST_request_scrub_priority: grep -v purged_snaps 2026-02-19T13:12:48.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6421: TEST_request_scrub_priority: head -1 2026-02-19T13:12:48.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6421: TEST_request_scrub_priority: sed 's/.*[[]DBG[]]//' 2026-02-19T13:12:48.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6421: TEST_request_scrub_priority: grep -q 1.1 2026-02-19T13:12:48.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T13:12:48.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:12:48.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:12:48.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:12:48.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:12:48.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:12:48.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:12:48.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:12:48.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:12:48.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:12:48.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:12:48.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:12:48.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:12:48.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:12:48.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:12:48.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:12:48.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:12:48.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:12:48.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:12:48.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:12:48.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:12:48.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:12:48.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:12:48.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:12:48.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:48.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:12:48.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:12:48.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:12:48.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:12:48.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T13:12:48.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T13:12:48.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T13:12:48.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T13:12:48.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:12:48.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:12:48.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:12:48.873 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:12:48.873 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:12:48.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:12:48.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:12:48.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:12:48.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:12:48.876 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:12:48.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:12:48.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:12:48.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:12:48.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:12:48.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:12:48.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:12:48.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:12:48.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:12:48.882 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:12:48.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:12:48.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:12:48.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:12:48.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:12:48.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:48.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:12:48.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:12:48.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:12:48.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:12:48.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T13:12:48.890 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:12:48.890 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:48.890 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:12:48.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T13:12:48.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:12:48.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:12:48.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:12:48.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T13:12:48.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_scrub_warning td/osd-scrub-repair 2026-02-19T13:12:48.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6011: TEST_scrub_warning: local dir=td/osd-scrub-repair 2026-02-19T13:12:48.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6012: TEST_scrub_warning: local poolname=psr_pool 2026-02-19T13:12:48.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6013: TEST_scrub_warning: local objname=POBJ 2026-02-19T13:12:48.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6014: TEST_scrub_warning: local scrubs=5 2026-02-19T13:12:48.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6015: TEST_scrub_warning: local deep_scrubs=5 2026-02-19T13:12:48.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6016: TEST_scrub_warning: local i1_day=86400 2026-02-19T13:12:48.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6017: TEST_scrub_warning: calc 86400 '*' 7 2026-02-19T13:12:48.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: calc: awk 'BEGIN{print 86400 * 7}' 2026-02-19T13:12:48.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6017: TEST_scrub_warning: local i7_days=604800 2026-02-19T13:12:48.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6018: TEST_scrub_warning: calc 86400 '*' 14 2026-02-19T13:12:48.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: calc: awk 'BEGIN{print 86400 * 14}' 2026-02-19T13:12:48.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6018: TEST_scrub_warning: local i14_days=1209600 2026-02-19T13:12:48.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6019: TEST_scrub_warning: local overdue=0.5 2026-02-19T13:12:48.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6020: TEST_scrub_warning: calc 604800 + 86400 + '(' 604800 '*' 0.5 ')' 2026-02-19T13:12:48.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: calc: awk 'BEGIN{print 604800 + 86400 + ( 604800 * 0.5 )}' 2026-02-19T13:12:48.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6020: TEST_scrub_warning: local conf_overdue_seconds=993600 2026-02-19T13:12:48.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6021: TEST_scrub_warning: calc 1209600 + 86400 + '(' 1209600 '*' 0.5 ')' 2026-02-19T13:12:48.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1580: calc: awk 'BEGIN{print 1209600 + 86400 + ( 1209600 * 0.5 )}' 2026-02-19T13:12:48.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6021: TEST_scrub_warning: local pool_overdue_seconds=1900800 2026-02-19T13:12:48.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6023: TEST_scrub_warning: run_mon td/osd-scrub-repair a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-19T13:12:48.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T13:12:48.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:12:48.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:12:48.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:12:48.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T13:12:48.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-19T13:12:48.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:12:48.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:12:48.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:12:48.928 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:12:48.929 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:48.929 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:12:48.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:12:48.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-19T13:12:48.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:12:48.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:12:48.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:12:48.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:12:48.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:12:48.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:12:48.969 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:12:48.969 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:12:48.970 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:12:48.970 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:12:48.970 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:48.970 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:12:48.970 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:12:48.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:12:48.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T13:12:49.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:12:49.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:12:49.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:12:49.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:12:49.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:12:49.044 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:12:49.044 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:12:49.044 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:12:49.044 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:12:49.044 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:49.045 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:12:49.045 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:12:49.045 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:12:49.045 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T13:12:49.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6024: TEST_scrub_warning: run_mgr td/osd-scrub-repair x --mon_warn_pg_not_scrubbed_ratio=0.5 --mon_warn_pg_not_deep_scrubbed_ratio=0.5 2026-02-19T13:12:49.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T13:12:49.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:12:49.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:12:49.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:12:49.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T13:12:49.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:12:49.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:12:49.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:12:49.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:12:49.197 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:12:49.197 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:49.197 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:12:49.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:12:49.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:12:49.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mon_warn_pg_not_scrubbed_ratio=0.5 --mon_warn_pg_not_deep_scrubbed_ratio=0.5 2026-02-19T13:12:49.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6025: TEST_scrub_warning: run_osd td/osd-scrub-repair 0 --osd_scrub_backoff_ratio=0 2026-02-19T13:12:49.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:12:49.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:12:49.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:12:49.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:12:49.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:12:49.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:12:49.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:12:49.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:12:49.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:12:49.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:12:49.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:12:49.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:12:49.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:12:49.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:12:49.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:12:49.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:12:49.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:12:49.224 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:12:49.224 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:49.224 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:12:49.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:12:49.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:12:49.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:12:49.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:12:49.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:12:49.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:12:49.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:12:49.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:12:49.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:12:49.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:12:49.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:12:49.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:12:49.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_scrub_backoff_ratio=0 2026-02-19T13:12:49.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:12:49.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:12:49.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2381f4ad-0ac3-4296-b95d-effc2a7fed30 2026-02-19T13:12:49.226 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 2381f4ad-0ac3-4296-b95d-effc2a7fed30 2026-02-19T13:12:49.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 2381f4ad-0ac3-4296-b95d-effc2a7fed30' 2026-02-19T13:12:49.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:12:49.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBRDJdpgY8cDhAA/tlUQDEXpv+NDsty4Ry9PA== 2026-02-19T13:12:49.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBRDJdpgY8cDhAA/tlUQDEXpv+NDsty4Ry9PA=="}' 2026-02-19T13:12:49.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2381f4ad-0ac3-4296-b95d-effc2a7fed30 -i td/osd-scrub-repair/0/new.json 2026-02-19T13:12:49.325 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:12:49.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T13:12:49.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_backoff_ratio=0 --mkfs --key AQBRDJdpgY8cDhAA/tlUQDEXpv+NDsty4Ry9PA== --osd-uuid 2381f4ad-0ac3-4296-b95d-effc2a7fed30 2026-02-19T13:12:49.350 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:12:49.348+0000 7fdd84f26a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:49.352 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:12:49.352+0000 7fdd84f26a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:49.353 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:12:49.352+0000 7fdd84f26a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:49.353 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:12:49.352+0000 7fdd84f26a40 -1 bdev(0x55e3ea900c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:12:49.353 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:12:49.352+0000 7fdd84f26a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T13:12:50.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T13:12:50.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:12:50.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:12:50.092 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:12:50.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:12:50.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:12:50.296 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:12:50.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_backoff_ratio=0 2026-02-19T13:12:50.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:12:50.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:12:50.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:12:50.321 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:12:50.320+0000 7efd41fdca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:50.323 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:12:50.324+0000 7efd41fdca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:50.324 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:12:50.324+0000 7efd41fdca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:50.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:12:50.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:12:50.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:12:50.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:12:50.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:12:50.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:12:50.460 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:12:50.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:12:50.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:12:50.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:12:50.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:12:50.733 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:12:50.732+0000 7efd41fdca40 -1 Falling back to public interface 2026-02-19T13:12:51.340 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:12:51.340+0000 7efd41fdca40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:12:51.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:12:51.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:12:51.621 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:12:51.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:12:51.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:12:51.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:12:51.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:12:52.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:12:52.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:12:52.785 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:12:52.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:12:52.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:12:52.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:12:52.946 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3311894773,v1:127.0.0.1:6803/3311894773] [v2:127.0.0.1:6804/3311894773,v1:127.0.0.1:6805/3311894773] exists,up 2381f4ad-0ac3-4296-b95d-effc2a7fed30 2026-02-19T13:12:52.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:12:52.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:12:52.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:12:52.947 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6027: TEST_scrub_warning: expr 5 + 5 2026-02-19T13:12:52.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6027: TEST_scrub_warning: seq 1 10 2026-02-19T13:12:52.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6027: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-19T13:12:52.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6029: TEST_scrub_warning: create_pool psr_pool-1 1 1 2026-02-19T13:12:52.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create psr_pool-1 1 1 2026-02-19T13:12:53.155 INFO:tasks.workunit.client.0.trial144.stderr:pool 'psr_pool-1' created 2026-02-19T13:12:53.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:12:54.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6030: TEST_scrub_warning: wait_for_clean 2026-02-19T13:12:54.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:12:54.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:12:54.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:12:54.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:12:54.169 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:12:54.169 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:12:54.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:12:54.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:12:54.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:12:54.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:12:54.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:12:54.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:12:54.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:12:54.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:12:54.287 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:12:54.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:12:54.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:12:54.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:54.452 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:12:54.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836482 2026-02-19T13:12:54.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836482 2026-02-19T13:12:54.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836482' 2026-02-19T13:12:54.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:54.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836482 2026-02-19T13:12:54.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:54.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:12:54.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836482 2026-02-19T13:12:54.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:54.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836482 2026-02-19T13:12:54.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836482' 2026-02-19T13:12:54.533 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836482 2026-02-19T13:12:54.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:12:54.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836482 2026-02-19T13:12:54.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:12:55.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:12:55.704 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:12:55.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836482 2026-02-19T13:12:55.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:12:55.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:12:55.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:12:56.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:12:56.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:12:56.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:12:56.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:12:56.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:12:56.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:12:56.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:12:56.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:12:56.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:12:56.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:12:56.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:12:56.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:12:56.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:12:56.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:12:56.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:12:56.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6031: TEST_scrub_warning: '[' 1 = 1 ']' 2026-02-19T13:12:56.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6033: TEST_scrub_warning: ceph osd pool set psr_pool-1 scrub_max_interval 1209600 2026-02-19T13:12:56.613 INFO:tasks.workunit.client.0.trial144.stderr:set pool 1 scrub_max_interval to 1209600 2026-02-19T13:12:56.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:12:56.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: '[' 1 = 6 ']' 2026-02-19T13:12:56.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6027: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-19T13:12:56.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6029: TEST_scrub_warning: create_pool psr_pool-2 1 1 2026-02-19T13:12:56.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create psr_pool-2 1 1 2026-02-19T13:12:56.818 INFO:tasks.workunit.client.0.trial144.stderr:pool 'psr_pool-2' created 2026-02-19T13:12:56.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:12:57.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6030: TEST_scrub_warning: wait_for_clean 2026-02-19T13:12:57.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:12:57.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:12:57.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:12:57.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:12:57.833 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:12:57.833 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:12:57.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:12:57.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:12:57.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:12:57.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:12:57.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:12:57.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:12:57.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:12:57.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:12:57.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:12:58.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:12:58.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:12:58.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:58.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:12:58.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-19T13:12:58.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-19T13:12:58.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-19T13:12:58.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:58.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-19T13:12:58.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:58.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:12:58.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-19T13:12:58.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:58.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-19T13:12:58.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-19T13:12:58.183 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836484 2026-02-19T13:12:58.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:12:58.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-19T13:12:58.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:12:59.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:12:59.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:12:59.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-19T13:12:59.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:00.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:13:00.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:00.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-19T13:13:00.665 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:13:00.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:00.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:00.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 2 == 0 2026-02-19T13:13:00.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:13:00.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:13:00.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:13:00.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:13:00.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:13:00.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:13:00.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:13:01.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=2 2026-02-19T13:13:01.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:13:01.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:01.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:01.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 2 = 2 2026-02-19T13:13:01.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:13:01.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:13:01.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6031: TEST_scrub_warning: '[' 2 = 1 ']' 2026-02-19T13:13:01.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:13:01.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: '[' 2 = 6 ']' 2026-02-19T13:13:01.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6027: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-19T13:13:01.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6029: TEST_scrub_warning: create_pool psr_pool-3 1 1 2026-02-19T13:13:01.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create psr_pool-3 1 1 2026-02-19T13:13:01.404 INFO:tasks.workunit.client.0.trial144.stderr:pool 'psr_pool-3' created 2026-02-19T13:13:01.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:13:02.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6030: TEST_scrub_warning: wait_for_clean 2026-02-19T13:13:02.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:13:02.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:13:02.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:13:02.417 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:13:02.418 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:13:02.418 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:13:02.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:13:02.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:13:02.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:13:02.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:13:02.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:13:02.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:13:02.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:13:02.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:13:02.539 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:13:02.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:13:02.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:13:02.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:02.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:13:02.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836486 2026-02-19T13:13:02.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836486 2026-02-19T13:13:02.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836486' 2026-02-19T13:13:02.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:02.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836486 2026-02-19T13:13:02.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:02.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:13:02.774 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836486 2026-02-19T13:13:02.774 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:02.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836486 2026-02-19T13:13:02.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-19T13:13:02.776 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836486 2026-02-19T13:13:02.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:02.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836486 2026-02-19T13:13:02.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:03.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:13:03.946 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:04.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-19T13:13:04.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:13:04.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:04.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:04.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 3 == 0 2026-02-19T13:13:04.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:13:04.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:13:04.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:13:04.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:13:04.298 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:13:04.298 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:13:04.298 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:13:04.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=3 2026-02-19T13:13:04.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:13:04.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:04.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:04.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 3 = 3 2026-02-19T13:13:04.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:13:04.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:13:04.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6031: TEST_scrub_warning: '[' 3 = 1 ']' 2026-02-19T13:13:04.637 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:13:04.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: '[' 3 = 6 ']' 2026-02-19T13:13:04.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6027: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-19T13:13:04.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6029: TEST_scrub_warning: create_pool psr_pool-4 1 1 2026-02-19T13:13:04.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create psr_pool-4 1 1 2026-02-19T13:13:04.841 INFO:tasks.workunit.client.0.trial144.stderr:pool 'psr_pool-4' created 2026-02-19T13:13:04.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:13:05.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6030: TEST_scrub_warning: wait_for_clean 2026-02-19T13:13:05.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:13:05.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:13:05.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:13:05.856 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:13:05.857 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:13:05.857 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:13:05.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:13:05.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:13:05.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:13:05.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:13:05.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:13:05.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:13:05.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:13:05.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:13:05.983 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:13:06.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:13:06.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:13:06.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:06.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:13:06.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-19T13:13:06.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-19T13:13:06.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-19T13:13:06.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:06.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-19T13:13:06.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:06.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:13:06.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-19T13:13:06.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:06.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-19T13:13:06.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-19T13:13:06.223 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836487 2026-02-19T13:13:06.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:06.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T13:13:06.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:07.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:13:07.385 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:07.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T13:13:07.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:08.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:13:08.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:08.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836487 2026-02-19T13:13:08.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:13:08.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:08.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:08.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T13:13:08.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:13:08.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:13:08.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:13:08.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:13:08.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:13:08.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:13:08.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:13:09.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T13:13:09.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:13:09.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:09.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:09.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T13:13:09.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:13:09.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:13:09.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6031: TEST_scrub_warning: '[' 4 = 1 ']' 2026-02-19T13:13:09.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:13:09.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: '[' 4 = 6 ']' 2026-02-19T13:13:09.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6027: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-19T13:13:09.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6029: TEST_scrub_warning: create_pool psr_pool-5 1 1 2026-02-19T13:13:09.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create psr_pool-5 1 1 2026-02-19T13:13:09.470 INFO:tasks.workunit.client.0.trial144.stderr:pool 'psr_pool-5' created 2026-02-19T13:13:09.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:13:10.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6030: TEST_scrub_warning: wait_for_clean 2026-02-19T13:13:10.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:13:10.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:13:10.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:13:10.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:13:10.484 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:13:10.484 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:13:10.485 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:13:10.485 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:13:10.485 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:13:10.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:13:10.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:13:10.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:13:10.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:13:10.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:13:10.605 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:13:10.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:13:10.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:13:10.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:10.766 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:13:10.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836489 2026-02-19T13:13:10.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836489 2026-02-19T13:13:10.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489' 2026-02-19T13:13:10.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:10.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836489 2026-02-19T13:13:10.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:10.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:13:10.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836489 2026-02-19T13:13:10.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:10.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836489 2026-02-19T13:13:10.842 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836489 2026-02-19T13:13:10.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-19T13:13:10.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:11.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-19T13:13:11.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:12.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:13:12.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:12.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-19T13:13:12.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:13:12.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:12.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:12.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:13:12.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:13:12.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:13:12.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:13:12.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:13:12.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:13:12.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:13:12.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:13:12.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:13:12.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:13:12.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:12.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:12.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:13:12.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:13:12.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:13:12.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6031: TEST_scrub_warning: '[' 5 = 1 ']' 2026-02-19T13:13:12.713 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:13:12.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: '[' 5 = 6 ']' 2026-02-19T13:13:12.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6027: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-19T13:13:12.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6029: TEST_scrub_warning: create_pool psr_pool-6 1 1 2026-02-19T13:13:12.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create psr_pool-6 1 1 2026-02-19T13:13:12.916 INFO:tasks.workunit.client.0.trial144.stderr:pool 'psr_pool-6' created 2026-02-19T13:13:12.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:13:13.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6030: TEST_scrub_warning: wait_for_clean 2026-02-19T13:13:13.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:13:13.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:13:13.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:13:13.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:13:13.930 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:13:13.930 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:13:13.930 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:13:13.930 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:13:13.931 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:13:14.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:13:14.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:13:14.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:13:14.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:13:14.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:13:14.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:13:14.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:13:14.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:13:14.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:14.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:13:14.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-19T13:13:14.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-19T13:13:14.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491' 2026-02-19T13:13:14.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:14.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-19T13:13:14.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:14.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:13:14.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836491 2026-02-19T13:13:14.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:14.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836491 2026-02-19T13:13:14.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-19T13:13:14.284 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836491 2026-02-19T13:13:14.285 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:14.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T13:13:14.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:15.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:13:15.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:15.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T13:13:15.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:16.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:13:16.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:16.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-19T13:13:16.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:13:16.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:16.762 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:16.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 6 == 0 2026-02-19T13:13:16.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:13:16.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:13:16.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:13:16.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:13:16.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:13:16.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:13:16.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:13:17.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=6 2026-02-19T13:13:17.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:13:17.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:17.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:17.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 = 6 2026-02-19T13:13:17.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:13:17.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:13:17.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6031: TEST_scrub_warning: '[' 6 = 1 ']' 2026-02-19T13:13:17.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:13:17.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: '[' 6 = 6 ']' 2026-02-19T13:13:17.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6037: TEST_scrub_warning: ceph osd pool set psr_pool-6 deep_scrub_interval 1209600 2026-02-19T13:13:17.477 INFO:tasks.workunit.client.0.trial144.stderr:set pool 6 deep_scrub_interval to 1209600 2026-02-19T13:13:17.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6027: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-19T13:13:17.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6029: TEST_scrub_warning: create_pool psr_pool-7 1 1 2026-02-19T13:13:17.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create psr_pool-7 1 1 2026-02-19T13:13:17.680 INFO:tasks.workunit.client.0.trial144.stderr:pool 'psr_pool-7' created 2026-02-19T13:13:17.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:13:18.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6030: TEST_scrub_warning: wait_for_clean 2026-02-19T13:13:18.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:13:18.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:13:18.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:13:18.693 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:13:18.694 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:13:18.694 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:13:18.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:13:18.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:13:18.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:13:18.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:13:18.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:13:18.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:13:18.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:13:18.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:13:18.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:13:18.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:13:18.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:13:18.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:18.985 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:13:19.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836493 2026-02-19T13:13:19.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836493 2026-02-19T13:13:19.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493' 2026-02-19T13:13:19.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:19.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836493 2026-02-19T13:13:19.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:19.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:13:19.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836493 2026-02-19T13:13:19.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:19.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836493 2026-02-19T13:13:19.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-19T13:13:19.063 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836493 2026-02-19T13:13:19.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:19.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836492 -lt 21474836493 2026-02-19T13:13:19.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:20.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:13:20.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:20.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836493 -lt 21474836493 2026-02-19T13:13:20.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:13:20.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:20.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:20.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 7 == 0 2026-02-19T13:13:20.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:13:20.575 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:13:20.575 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:13:20.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:13:20.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:13:20.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:13:20.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:13:20.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=7 2026-02-19T13:13:20.729 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:13:20.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:20.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:20.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 7 = 7 2026-02-19T13:13:20.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:13:20.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:13:20.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6031: TEST_scrub_warning: '[' 7 = 1 ']' 2026-02-19T13:13:20.920 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:13:20.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: '[' 7 = 6 ']' 2026-02-19T13:13:20.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6027: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-19T13:13:20.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6029: TEST_scrub_warning: create_pool psr_pool-8 1 1 2026-02-19T13:13:20.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create psr_pool-8 1 1 2026-02-19T13:13:21.130 INFO:tasks.workunit.client.0.trial144.stderr:pool 'psr_pool-8' created 2026-02-19T13:13:21.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:13:22.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6030: TEST_scrub_warning: wait_for_clean 2026-02-19T13:13:22.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:13:22.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:13:22.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:13:22.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:13:22.146 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:13:22.146 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:13:22.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:13:22.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:13:22.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:13:22.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:13:22.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:13:22.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:13:22.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:13:22.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:13:22.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:13:22.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:13:22.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:13:22.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:22.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:13:22.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836494 2026-02-19T13:13:22.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836494 2026-02-19T13:13:22.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836494' 2026-02-19T13:13:22.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:22.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836494 2026-02-19T13:13:22.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:22.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:13:22.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836494 2026-02-19T13:13:22.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:22.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836494 2026-02-19T13:13:22.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836494' 2026-02-19T13:13:22.508 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836494 2026-02-19T13:13:22.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:22.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836493 -lt 21474836494 2026-02-19T13:13:22.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:23.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:13:23.673 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:23.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836495 -lt 21474836494 2026-02-19T13:13:23.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:13:23.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:23.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:24.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 8 == 0 2026-02-19T13:13:24.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:13:24.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:13:24.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:13:24.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:13:24.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:13:24.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:13:24.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:13:24.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=8 2026-02-19T13:13:24.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:13:24.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:24.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:24.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 8 = 8 2026-02-19T13:13:24.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:13:24.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:13:24.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6031: TEST_scrub_warning: '[' 8 = 1 ']' 2026-02-19T13:13:24.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:13:24.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: '[' 8 = 6 ']' 2026-02-19T13:13:24.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6027: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-19T13:13:24.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6029: TEST_scrub_warning: create_pool psr_pool-9 1 1 2026-02-19T13:13:24.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create psr_pool-9 1 1 2026-02-19T13:13:24.577 INFO:tasks.workunit.client.0.trial144.stderr:pool 'psr_pool-9' created 2026-02-19T13:13:24.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:13:25.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6030: TEST_scrub_warning: wait_for_clean 2026-02-19T13:13:25.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:13:25.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:13:25.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:13:25.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:13:25.593 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:13:25.593 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:13:25.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:13:25.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:13:25.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:13:25.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:13:25.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:13:25.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:13:25.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:13:25.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:13:25.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:13:25.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:13:25.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:13:25.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:25.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:13:25.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836496 2026-02-19T13:13:25.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836496 2026-02-19T13:13:25.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836496' 2026-02-19T13:13:25.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:25.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836496 2026-02-19T13:13:25.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:25.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:13:25.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836496 2026-02-19T13:13:25.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:25.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836496 2026-02-19T13:13:25.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836496' 2026-02-19T13:13:25.965 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836496 2026-02-19T13:13:25.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:26.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836495 -lt 21474836496 2026-02-19T13:13:26.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:27.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:13:27.136 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:27.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836495 -lt 21474836496 2026-02-19T13:13:27.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:28.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:13:28.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:28.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836497 -lt 21474836496 2026-02-19T13:13:28.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:13:28.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:28.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:28.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 9 == 0 2026-02-19T13:13:28.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:13:28.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:13:28.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:13:28.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:13:28.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:13:28.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:13:28.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:13:28.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=9 2026-02-19T13:13:28.806 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:13:28.807 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:28.807 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:29.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 9 = 9 2026-02-19T13:13:29.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:13:29.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:13:29.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6031: TEST_scrub_warning: '[' 9 = 1 ']' 2026-02-19T13:13:29.001 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:13:29.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: '[' 9 = 6 ']' 2026-02-19T13:13:29.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6027: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-19T13:13:29.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6029: TEST_scrub_warning: create_pool psr_pool-10 1 1 2026-02-19T13:13:29.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create psr_pool-10 1 1 2026-02-19T13:13:29.207 INFO:tasks.workunit.client.0.trial144.stderr:pool 'psr_pool-10' created 2026-02-19T13:13:29.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:13:30.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6030: TEST_scrub_warning: wait_for_clean 2026-02-19T13:13:30.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:13:30.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:13:30.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:13:30.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:13:30.221 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:13:30.222 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:13:30.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:13:30.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:13:30.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:13:30.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:13:30.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:13:30.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:13:30.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:13:30.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:13:30.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:13:30.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:13:30.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:13:30.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:30.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:13:30.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836498 2026-02-19T13:13:30.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836498 2026-02-19T13:13:30.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836498' 2026-02-19T13:13:30.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:30.573 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836498 2026-02-19T13:13:30.574 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:30.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:13:30.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836498 2026-02-19T13:13:30.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836498 2026-02-19T13:13:30.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2026-02-19T13:13:30.578 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836498 2026-02-19T13:13:30.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:30.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836497 -lt 21474836498 2026-02-19T13:13:30.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:31.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:13:31.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:31.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836498 -lt 21474836498 2026-02-19T13:13:31.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:13:31.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:31.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:32.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 10 == 0 2026-02-19T13:13:32.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:13:32.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:13:32.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:13:32.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:13:32.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:13:32.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:13:32.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:13:32.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=10 2026-02-19T13:13:32.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:13:32.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:32.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:32.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 10 = 10 2026-02-19T13:13:32.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:13:32.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:13:32.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6031: TEST_scrub_warning: '[' 10 = 1 ']' 2026-02-19T13:13:32.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:13:32.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6035: TEST_scrub_warning: '[' 10 = 6 ']' 2026-02-19T13:13:32.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6042: TEST_scrub_warning: local primary=0 2026-02-19T13:13:32.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6044: TEST_scrub_warning: ceph osd set noscrub 2026-02-19T13:13:32.645 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:13:32.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6045: TEST_scrub_warning: ceph osd set nodeep-scrub 2026-02-19T13:13:32.849 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:13:32.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6046: TEST_scrub_warning: ceph config set global osd_scrub_interval_randomize_ratio 0 2026-02-19T13:13:33.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6047: TEST_scrub_warning: ceph config set global osd_deep_scrub_randomize_ratio 0 2026-02-19T13:13:33.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6048: TEST_scrub_warning: ceph config set global osd_scrub_max_interval 604800 2026-02-19T13:13:33.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6049: TEST_scrub_warning: ceph config set global osd_deep_scrub_interval 604800 2026-02-19T13:13:33.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6052: TEST_scrub_warning: seq 1 5 2026-02-19T13:13:33.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6052: TEST_scrub_warning: for i in $(seq 1 $scrubs) 2026-02-19T13:13:33.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6054: TEST_scrub_warning: '[' 1 = 1 ']' 2026-02-19T13:13:33.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6056: TEST_scrub_warning: overdue_seconds=1900800 2026-02-19T13:13:33.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6060: TEST_scrub_warning: expr 1900800 + 100 2026-02-19T13:13:33.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6060: TEST_scrub_warning: ceph tell 1.0 schedule-scrub 1900900 2026-02-19T13:13:33.597 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:13:33.597 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:13:33.597 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:13:33.597 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-01-28T13:11:53.597587+0000" 2026-02-19T13:13:33.597 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:13:33.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6052: TEST_scrub_warning: for i in $(seq 1 $scrubs) 2026-02-19T13:13:33.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6054: TEST_scrub_warning: '[' 2 = 1 ']' 2026-02-19T13:13:33.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6058: TEST_scrub_warning: overdue_seconds=993600 2026-02-19T13:13:33.605 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6060: TEST_scrub_warning: expr 993600 + 200 2026-02-19T13:13:33.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6060: TEST_scrub_warning: ceph tell 2.0 schedule-scrub 993800 2026-02-19T13:13:33.673 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:13:33.673 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:13:33.673 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:13:33.673 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-08T01:10:13.673795+0000" 2026-02-19T13:13:33.673 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:13:33.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6052: TEST_scrub_warning: for i in $(seq 1 $scrubs) 2026-02-19T13:13:33.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6054: TEST_scrub_warning: '[' 3 = 1 ']' 2026-02-19T13:13:33.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6058: TEST_scrub_warning: overdue_seconds=993600 2026-02-19T13:13:33.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6060: TEST_scrub_warning: expr 993600 + 300 2026-02-19T13:13:33.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6060: TEST_scrub_warning: ceph tell 3.0 schedule-scrub 993900 2026-02-19T13:13:33.747 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:13:33.747 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:13:33.747 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:13:33.747 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-08T01:08:33.747925+0000" 2026-02-19T13:13:33.747 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:13:33.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6052: TEST_scrub_warning: for i in $(seq 1 $scrubs) 2026-02-19T13:13:33.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6054: TEST_scrub_warning: '[' 4 = 1 ']' 2026-02-19T13:13:33.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6058: TEST_scrub_warning: overdue_seconds=993600 2026-02-19T13:13:33.758 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6060: TEST_scrub_warning: expr 993600 + 400 2026-02-19T13:13:33.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6060: TEST_scrub_warning: ceph tell 4.0 schedule-scrub 994000 2026-02-19T13:13:33.825 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:13:33.825 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:13:33.825 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:13:33.825 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-08T01:06:53.826032+0000" 2026-02-19T13:13:33.825 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:13:33.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6052: TEST_scrub_warning: for i in $(seq 1 $scrubs) 2026-02-19T13:13:33.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6054: TEST_scrub_warning: '[' 5 = 1 ']' 2026-02-19T13:13:33.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6058: TEST_scrub_warning: overdue_seconds=993600 2026-02-19T13:13:33.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6060: TEST_scrub_warning: expr 993600 + 500 2026-02-19T13:13:33.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6060: TEST_scrub_warning: ceph tell 5.0 schedule-scrub 994100 2026-02-19T13:13:33.901 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:13:33.901 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:13:33.901 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:13:33.901 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-08T01:05:13.901731+0000" 2026-02-19T13:13:33.901 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:13:33.912 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6063: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:13:33.913 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6063: TEST_scrub_warning: expr 5 + 5 2026-02-19T13:13:33.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6063: TEST_scrub_warning: seq 6 10 2026-02-19T13:13:33.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6063: TEST_scrub_warning: for i in $(seq $(expr $scrubs + 1) $(expr $scrubs + $deep_scrubs)) 2026-02-19T13:13:33.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6065: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:13:33.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6065: TEST_scrub_warning: '[' 6 = 6 ']' 2026-02-19T13:13:33.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6067: TEST_scrub_warning: overdue_seconds=1900800 2026-02-19T13:13:33.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6071: TEST_scrub_warning: expr 1900800 + 600 2026-02-19T13:13:33.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6071: TEST_scrub_warning: ceph tell 6.0 schedule-deep-scrub 1901400 2026-02-19T13:13:33.986 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:13:33.986 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T13:13:33.987 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:13:33.987 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-01-28T13:03:33.987208+0000" 2026-02-19T13:13:33.987 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:13:33.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6063: TEST_scrub_warning: for i in $(seq $(expr $scrubs + 1) $(expr $scrubs + $deep_scrubs)) 2026-02-19T13:13:33.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6065: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:13:33.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6065: TEST_scrub_warning: '[' 7 = 6 ']' 2026-02-19T13:13:33.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6069: TEST_scrub_warning: overdue_seconds=993600 2026-02-19T13:13:33.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6071: TEST_scrub_warning: expr 993600 + 700 2026-02-19T13:13:33.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6071: TEST_scrub_warning: ceph tell 7.0 schedule-deep-scrub 994300 2026-02-19T13:13:34.061 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:13:34.062 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T13:13:34.062 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:13:34.062 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-08T01:01:54.062463+0000" 2026-02-19T13:13:34.062 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:13:34.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6063: TEST_scrub_warning: for i in $(seq $(expr $scrubs + 1) $(expr $scrubs + $deep_scrubs)) 2026-02-19T13:13:34.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6065: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:13:34.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6065: TEST_scrub_warning: '[' 8 = 6 ']' 2026-02-19T13:13:34.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6069: TEST_scrub_warning: overdue_seconds=993600 2026-02-19T13:13:34.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6071: TEST_scrub_warning: expr 993600 + 800 2026-02-19T13:13:34.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6071: TEST_scrub_warning: ceph tell 8.0 schedule-deep-scrub 994400 2026-02-19T13:13:34.137 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:13:34.137 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T13:13:34.137 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:13:34.137 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-08T01:00:14.137985+0000" 2026-02-19T13:13:34.137 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:13:34.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6063: TEST_scrub_warning: for i in $(seq $(expr $scrubs + 1) $(expr $scrubs + $deep_scrubs)) 2026-02-19T13:13:34.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6065: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:13:34.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6065: TEST_scrub_warning: '[' 9 = 6 ']' 2026-02-19T13:13:34.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6069: TEST_scrub_warning: overdue_seconds=993600 2026-02-19T13:13:34.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6071: TEST_scrub_warning: expr 993600 + 900 2026-02-19T13:13:34.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6071: TEST_scrub_warning: ceph tell 9.0 schedule-deep-scrub 994500 2026-02-19T13:13:34.216 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:13:34.216 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T13:13:34.216 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:13:34.216 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-08T00:58:34.216973+0000" 2026-02-19T13:13:34.216 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:13:34.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6063: TEST_scrub_warning: for i in $(seq $(expr $scrubs + 1) $(expr $scrubs + $deep_scrubs)) 2026-02-19T13:13:34.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6065: TEST_scrub_warning: expr 5 + 1 2026-02-19T13:13:34.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6065: TEST_scrub_warning: '[' 10 = 6 ']' 2026-02-19T13:13:34.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6069: TEST_scrub_warning: overdue_seconds=993600 2026-02-19T13:13:34.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6071: TEST_scrub_warning: expr 993600 + 1000 2026-02-19T13:13:34.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6071: TEST_scrub_warning: ceph tell 10.0 schedule-deep-scrub 994600 2026-02-19T13:13:34.291 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:13:34.292 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T13:13:34.292 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:13:34.292 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-08T00:56:54.292484+0000" 2026-02-19T13:13:34.292 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:13:34.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6073: TEST_scrub_warning: flush_pg_stats 2026-02-19T13:13:34.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:13:34.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:13:34.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:13:34.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:13:34.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:34.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:13:34.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836500 2026-02-19T13:13:34.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836500 2026-02-19T13:13:34.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836500' 2026-02-19T13:13:34.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:34.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836500 2026-02-19T13:13:34.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:34.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:13:34.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836500 2026-02-19T13:13:34.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:34.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836500 2026-02-19T13:13:34.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836500' 2026-02-19T13:13:34.548 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836500 2026-02-19T13:13:34.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:34.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836499 -lt 21474836500 2026-02-19T13:13:34.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:35.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:13:35.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:35.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836500 -lt 21474836500 2026-02-19T13:13:35.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6075: TEST_scrub_warning: ceph health 2026-02-19T13:13:36.040 INFO:tasks.workunit.client.0.trial144.stdout:HEALTH_WARN noscrub,nodeep-scrub flag(s) set; 5 pgs not deep-scrubbed in time; 5 pgs not scrubbed in time; 10 pool(s) have no replicas configured; OSD count 1 < osd_pool_default_size 3 2026-02-19T13:13:36.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6076: TEST_scrub_warning: ceph health detail 2026-02-19T13:13:36.232 INFO:tasks.workunit.client.0.trial144.stdout:HEALTH_WARN noscrub,nodeep-scrub flag(s) set; 5 pgs not deep-scrubbed in time; 5 pgs not scrubbed in time; 10 pool(s) have no replicas configured; OSD count 1 < osd_pool_default_size 3 2026-02-19T13:13:36.232 INFO:tasks.workunit.client.0.trial144.stdout:[WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2026-02-19T13:13:36.232 INFO:tasks.workunit.client.0.trial144.stdout:[WRN] PG_NOT_DEEP_SCRUBBED: 5 pgs not deep-scrubbed in time 2026-02-19T13:13:36.233 INFO:tasks.workunit.client.0.trial144.stdout: pg 10.0 not deep-scrubbed since 2026-02-08T00:56:54.292484+0000 2026-02-19T13:13:36.233 INFO:tasks.workunit.client.0.trial144.stdout: pg 9.0 not deep-scrubbed since 2026-02-08T00:58:34.216973+0000 2026-02-19T13:13:36.233 INFO:tasks.workunit.client.0.trial144.stdout: pg 8.0 not deep-scrubbed since 2026-02-08T01:00:14.137985+0000 2026-02-19T13:13:36.233 INFO:tasks.workunit.client.0.trial144.stdout: pg 7.0 not deep-scrubbed since 2026-02-08T01:01:54.062463+0000 2026-02-19T13:13:36.233 INFO:tasks.workunit.client.0.trial144.stdout: pg 6.0 not deep-scrubbed since 2026-01-28T13:03:33.987208+0000 2026-02-19T13:13:36.233 INFO:tasks.workunit.client.0.trial144.stdout:[WRN] PG_NOT_SCRUBBED: 5 pgs not scrubbed in time 2026-02-19T13:13:36.233 INFO:tasks.workunit.client.0.trial144.stdout: pg 5.0 not scrubbed since 2026-02-08T01:05:13.901731+0000 2026-02-19T13:13:36.233 INFO:tasks.workunit.client.0.trial144.stdout: pg 4.0 not scrubbed since 2026-02-08T01:06:53.826032+0000 2026-02-19T13:13:36.234 INFO:tasks.workunit.client.0.trial144.stdout: pg 3.0 not scrubbed since 2026-02-08T01:08:33.747925+0000 2026-02-19T13:13:36.234 INFO:tasks.workunit.client.0.trial144.stdout: pg 2.0 not scrubbed since 2026-02-08T01:10:13.673795+0000 2026-02-19T13:13:36.234 INFO:tasks.workunit.client.0.trial144.stdout: pg 1.0 not scrubbed since 2026-01-28T13:11:53.597587+0000 2026-02-19T13:13:36.234 INFO:tasks.workunit.client.0.trial144.stdout:[WRN] POOL_NO_REDUNDANCY: 10 pool(s) have no replicas configured 2026-02-19T13:13:36.234 INFO:tasks.workunit.client.0.trial144.stdout: pool 'psr_pool-1' has no replicas configured 2026-02-19T13:13:36.234 INFO:tasks.workunit.client.0.trial144.stdout: pool 'psr_pool-2' has no replicas configured 2026-02-19T13:13:36.234 INFO:tasks.workunit.client.0.trial144.stdout: pool 'psr_pool-3' has no replicas configured 2026-02-19T13:13:36.234 INFO:tasks.workunit.client.0.trial144.stdout: pool 'psr_pool-4' has no replicas configured 2026-02-19T13:13:36.235 INFO:tasks.workunit.client.0.trial144.stdout: pool 'psr_pool-5' has no replicas configured 2026-02-19T13:13:36.235 INFO:tasks.workunit.client.0.trial144.stdout: pool 'psr_pool-6' has no replicas configured 2026-02-19T13:13:36.235 INFO:tasks.workunit.client.0.trial144.stdout: pool 'psr_pool-7' has no replicas configured 2026-02-19T13:13:36.235 INFO:tasks.workunit.client.0.trial144.stdout: pool 'psr_pool-8' has no replicas configured 2026-02-19T13:13:36.235 INFO:tasks.workunit.client.0.trial144.stdout: pool 'psr_pool-9' has no replicas configured 2026-02-19T13:13:36.235 INFO:tasks.workunit.client.0.trial144.stdout: pool 'psr_pool-10' has no replicas configured 2026-02-19T13:13:36.235 INFO:tasks.workunit.client.0.trial144.stdout:[WRN] TOO_FEW_OSDS: OSD count 1 < osd_pool_default_size 3 2026-02-19T13:13:36.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6077: TEST_scrub_warning: ceph health 2026-02-19T13:13:36.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6077: TEST_scrub_warning: grep -q ' pgs not deep-scrubbed in time' 2026-02-19T13:13:36.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6078: TEST_scrub_warning: ceph health 2026-02-19T13:13:36.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6078: TEST_scrub_warning: grep -q ' pgs not scrubbed in time' 2026-02-19T13:13:36.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6083: TEST_scrub_warning: ceph health detail 2026-02-19T13:13:36.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6083: TEST_scrub_warning: grep 'not scrubbed since' 2026-02-19T13:13:36.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6083: TEST_scrub_warning: wc -l 2026-02-19T13:13:36.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6083: TEST_scrub_warning: COUNT=5 2026-02-19T13:13:36.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6085: TEST_scrub_warning: expr 5+5 2026-02-19T13:13:36.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6085: TEST_scrub_warning: (( 5 != 5 && 5 != 5+5 )) 2026-02-19T13:13:36.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6089: TEST_scrub_warning: ceph health detail 2026-02-19T13:13:36.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6089: TEST_scrub_warning: grep 'not deep-scrubbed since' 2026-02-19T13:13:36.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6089: TEST_scrub_warning: wc -l 2026-02-19T13:13:37.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6089: TEST_scrub_warning: COUNT=5 2026-02-19T13:13:37.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6090: TEST_scrub_warning: '[' 5 '!=' 5 ']' 2026-02-19T13:13:37.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T13:13:37.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:13:37.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:13:37.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:13:37.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:13:37.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:13:37.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:13:37.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:13:37.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:13:37.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:13:37.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:13:37.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:13:37.131 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:13:37.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:13:37.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:13:37.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:13:37.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:13:37.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:13:37.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:13:37.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:13:37.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:13:37.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:13:37.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:13:37.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:13:37.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:37.146 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:13:37.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:13:37.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:13:37.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:13:37.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T13:13:37.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T13:13:37.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T13:13:37.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T13:13:37.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:13:37.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:13:37.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:13:37.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:13:37.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:13:37.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:13:37.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:13:37.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:13:37.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:13:37.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:13:37.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:13:37.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:13:37.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:13:37.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:13:37.155 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:13:37.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:13:37.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:13:37.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:13:37.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:13:37.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:13:37.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:13:37.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:13:37.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:13:37.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:37.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:13:37.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:13:37.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:13:37.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:13:37.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T13:13:37.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:13:37.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:37.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:13:37.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T13:13:37.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:13:37.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:13:37.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:13:37.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T13:13:37.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_skip_non_repair_during_recovery td/osd-scrub-repair 2026-02-19T13:13:37.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:160: TEST_skip_non_repair_during_recovery: local dir=td/osd-scrub-repair 2026-02-19T13:13:37.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:161: TEST_skip_non_repair_during_recovery: cluster_conf=(['osds_num']='2' ['pgs_in_pool']='4' ['pool_name']='nopool' ['pool_default_size']='2' ['extras']='--osd_scrub_during_recovery=false --osd_debug_pretend_recovery_active=true') 2026-02-19T13:13:37.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:161: TEST_skip_non_repair_during_recovery: local -A cluster_conf 2026-02-19T13:13:37.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: TEST_skip_non_repair_during_recovery: standard_scrub_cluster td/osd-scrub-repair cluster_conf 2026-02-19T13:13:37.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:229: standard_scrub_cluster: local dir=td/osd-scrub-repair 2026-02-19T13:13:37.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:230: standard_scrub_cluster: local -n args=cluster_conf 2026-02-19T13:13:37.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:232: standard_scrub_cluster: local OSDS=2 2026-02-19T13:13:37.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:233: standard_scrub_cluster: local pg_num=4 2026-02-19T13:13:37.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:235: standard_scrub_cluster: local poolname=nopool 2026-02-19T13:13:37.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:236: standard_scrub_cluster: args['pool_name']=nopool 2026-02-19T13:13:37.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:238: standard_scrub_cluster: local pool_default_size=2 2026-02-19T13:13:37.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:239: standard_scrub_cluster: args['pool_default_size']=2 2026-02-19T13:13:37.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:241: standard_scrub_cluster: local 'extra_pars=--osd_scrub_during_recovery=false --osd_debug_pretend_recovery_active=true' 2026-02-19T13:13:37.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:242: standard_scrub_cluster: local debug_msg=dbg 2026-02-19T13:13:37.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:245: standard_scrub_cluster: local saved_echo_flag=x 2026-02-19T13:13:37.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:246: standard_scrub_cluster: set +x 2026-02-19T13:13:37.455 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 d82e173b-028f-4e5d-85d6-d71c5f9427dd 2026-02-19T13:13:37.558 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:13:37.588 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:37.587+0000 7f3b5a86fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:37.590 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:37.587+0000 7f3b5a86fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:37.590 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:37.587+0000 7f3b5a86fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:37.591 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:37.587+0000 7f3b5a86fa40 -1 bdev(0x558a1142cc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:13:37.591 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:37.587+0000 7f3b5a86fa40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T13:13:38.392 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:13:38.601 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:13:38.622 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:38.619+0000 7fdb0af0da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:38.624 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:38.623+0000 7fdb0af0da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:38.625 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:38.623+0000 7fdb0af0da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:38.757 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:13:39.005 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:39.003+0000 7fdb0af0da40 -1 Falling back to public interface 2026-02-19T13:13:39.669 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:39.667+0000 7fdb0af0da40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:13:39.920 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:13:41.084 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:13:41.244 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/969800386,v1:127.0.0.1:6803/969800386] [v2:127.0.0.1:6804/969800386,v1:127.0.0.1:6805/969800386] exists,up d82e173b-028f-4e5d-85d6-d71c5f9427dd 2026-02-19T13:13:41.250 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 47560cbd-0cf9-407a-9e27-599bcc56a84a 2026-02-19T13:13:41.424 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:13:41.457 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:41.455+0000 7f22ba4dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:41.458 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:41.455+0000 7f22ba4dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:41.459 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:41.459+0000 7f22ba4dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:41.459 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:41.459+0000 7f22ba4dfa40 -1 bdev(0x55f472386c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:13:41.460 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:41.459+0000 7f22ba4dfa40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T13:13:42.099 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:13:42.304 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:13:42.337 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:42.335+0000 7f55abaefa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:42.338 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:42.335+0000 7f55abaefa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:42.339 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:42.339+0000 7f55abaefa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:42.460 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:13:42.717 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:42.715+0000 7f55abaefa40 -1 Falling back to public interface 2026-02-19T13:13:43.282 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:43.279+0000 7f55abaefa40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:13:43.625 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:13:44.788 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:13:44.951 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/790007536,v1:127.0.0.1:6811/790007536] [v2:127.0.0.1:6812/790007536,v1:127.0.0.1:6813/790007536] exists,up 47560cbd-0cf9-407a-9e27-599bcc56a84a 2026-02-19T13:13:45.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:170: TEST_skip_non_repair_during_recovery: local poolname=rbd 2026-02-19T13:13:45.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:171: TEST_skip_non_repair_during_recovery: create_rbd_pool 2026-02-19T13:13:45.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T13:13:46.107 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' does not exist 2026-02-19T13:13:46.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T13:13:46.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:13:46.338 INFO:tasks.workunit.client.0.trial144.stderr:pool 'rbd' created 2026-02-19T13:13:46.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:13:47.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T13:13:47.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:172: TEST_skip_non_repair_during_recovery: wait_for_clean 2026-02-19T13:13:47.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:13:47.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:13:47.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:13:47.635 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:13:47.635 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:13:47.635 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:13:47.635 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:13:47.635 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:13:47.635 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:13:47.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:13:47.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:13:47.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:13:47.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:13:47.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:13:47.755 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:13:47.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:13:47.910 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:13:47.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:13:47.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:47.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:13:47.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836489 2026-02-19T13:13:47.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836489 2026-02-19T13:13:47.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489' 2026-02-19T13:13:47.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:47.984 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:13:48.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672965 2026-02-19T13:13:48.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672965 2026-02-19T13:13:48.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-42949672965' 2026-02-19T13:13:48.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:48.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836489 2026-02-19T13:13:48.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:48.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:13:48.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836489 2026-02-19T13:13:48.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:48.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836489 2026-02-19T13:13:48.063 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836489 2026-02-19T13:13:48.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-19T13:13:48.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:48.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-19T13:13:48.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:49.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:13:49.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:49.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-19T13:13:49.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:50.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:13:50.385 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:50.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836491 -lt 21474836489 2026-02-19T13:13:50.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:50.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672965 2026-02-19T13:13:50.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:50.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:13:50.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672965 2026-02-19T13:13:50.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:50.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672965 2026-02-19T13:13:50.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672965' 2026-02-19T13:13:50.550 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672965 2026-02-19T13:13:50.550 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:13:50.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672967 -lt 42949672965 2026-02-19T13:13:50.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:13:50.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:50.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:50.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T13:13:50.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:13:50.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:13:50.919 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:13:50.919 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:13:50.919 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:13:50.919 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:13:50.919 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:13:51.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T13:13:51.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:13:51.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:51.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:51.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T13:13:51.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:13:51.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:13:51.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:174: TEST_skip_non_repair_during_recovery: add_something td/osd-scrub-repair rbd 2026-02-19T13:13:51.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:13:51.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=rbd 2026-02-19T13:13:51.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-19T13:13:51.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:13:51.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:13:51.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:13:51.476 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:13:51.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:13:51.681 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:13:51.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:13:51.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:13:51.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool rbd put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-19T13:13:51.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:175: TEST_skip_non_repair_during_recovery: get_not_primary rbd SOMETHING 2026-02-19T13:13:51.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=rbd 2026-02-19T13:13:51.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=SOMETHING 2026-02-19T13:13:51.718 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary rbd SOMETHING 2026-02-19T13:13:51.719 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-19T13:13:51.719 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T13:13:51.719 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd SOMETHING 2026-02-19T13:13:51.719 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:13:51.883 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T13:13:51.883 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map rbd SOMETHING 2026-02-19T13:13:51.884 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T13:13:52.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:175: TEST_skip_non_repair_during_recovery: scrub_and_not_schedule td/osd-scrub-repair rbd 0 2026-02-19T13:13:52.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:214: scrub_and_not_schedule: local dir=td/osd-scrub-repair 2026-02-19T13:13:52.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: scrub_and_not_schedule: local poolname=rbd 2026-02-19T13:13:52.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: scrub_and_not_schedule: local osd=0 2026-02-19T13:13:52.052 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:221: scrub_and_not_schedule: get_pg rbd SOMETHING 2026-02-19T13:13:52.052 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=rbd 2026-02-19T13:13:52.052 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T13:13:52.052 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map rbd SOMETHING 2026-02-19T13:13:52.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:13:52.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:221: scrub_and_not_schedule: local pg=1.3 2026-02-19T13:13:52.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:222: scrub_and_not_schedule: get_last_scrub_stamp 1.3 2026-02-19T13:13:52.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T13:13:52.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:13:52.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:13:52.212 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T13:13:52.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:222: scrub_and_not_schedule: local last_scrub=2026-02-19T13:13:46.338456+0000 2026-02-19T13:13:52.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:223: scrub_and_not_schedule: ceph tell 1.3 schedule-scrub 2026-02-19T13:13:52.436 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:13:52.437 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:13:52.437 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:13:52.437 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-17T13:13:52.437386+0000" 2026-02-19T13:13:52.437 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:13:52.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: scrub_and_not_schedule: sleep 3 2026-02-19T13:13:55.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:229: scrub_and_not_schedule: (( i=0 )) 2026-02-19T13:13:55.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:229: scrub_and_not_schedule: (( i < 3 )) 2026-02-19T13:13:55.446 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:230: scrub_and_not_schedule: get_last_scrub_stamp 1.3 2026-02-19T13:13:55.446 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T13:13:55.446 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:13:55.446 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:13:55.447 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T13:13:55.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:230: scrub_and_not_schedule: test 2026-02-17T13:13:52.437386+0000 '>' 2026-02-19T13:13:46.338456+0000 2026-02-19T13:13:55.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: scrub_and_not_schedule: sleep 1 2026-02-19T13:13:56.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:229: scrub_and_not_schedule: (( i++ )) 2026-02-19T13:13:56.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:229: scrub_and_not_schedule: (( i < 3 )) 2026-02-19T13:13:56.607 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:230: scrub_and_not_schedule: get_last_scrub_stamp 1.3 2026-02-19T13:13:56.608 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T13:13:56.608 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:13:56.608 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T13:13:56.608 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:13:56.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:230: scrub_and_not_schedule: test 2026-02-17T13:13:52.437386+0000 '>' 2026-02-19T13:13:46.338456+0000 2026-02-19T13:13:56.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: scrub_and_not_schedule: sleep 1 2026-02-19T13:13:57.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:229: scrub_and_not_schedule: (( i++ )) 2026-02-19T13:13:57.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:229: scrub_and_not_schedule: (( i < 3 )) 2026-02-19T13:13:57.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:230: scrub_and_not_schedule: get_last_scrub_stamp 1.3 2026-02-19T13:13:57.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.3 2026-02-19T13:13:57.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:13:57.770 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:13:57.770 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-19T13:13:57.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:230: scrub_and_not_schedule: test 2026-02-17T13:13:52.437386+0000 '>' 2026-02-19T13:13:46.338456+0000 2026-02-19T13:13:57.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: scrub_and_not_schedule: sleep 1 2026-02-19T13:13:58.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:229: scrub_and_not_schedule: (( i++ )) 2026-02-19T13:13:58.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:229: scrub_and_not_schedule: (( i < 3 )) 2026-02-19T13:13:58.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:239: scrub_and_not_schedule: objectstore_tool td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-19T13:13:58.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:13:58.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:13:58.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:13:58.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:13:58.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-19T13:13:58.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:13:58.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:13:58.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:13:58.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:13:58.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:13:58.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:13:58.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:13:58.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:13:58.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:13:58.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:13:59.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:13:59.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-19T13:13:59.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:13:59.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:13:59.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:13:59.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:13:59.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:13:59.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING list-attrs 2026-02-19T13:13:59.317 INFO:tasks.workunit.client.0.trial144.stdout:_ 2026-02-19T13:13:59.317 INFO:tasks.workunit.client.0.trial144.stdout:snapset 2026-02-19T13:13:59.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:13:59.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:13:59.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:13:59.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:13:59.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:13:59.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:13:59.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:13:59.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:13:59.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:13:59.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:13:59.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:13:59.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:13:59.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:13:59.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:13:59.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:13:59.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:13:59.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:13:59.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:13:59.413 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:13:59.413 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:59.413 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:13:59.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:13:59.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:13:59.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:13:59.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:13:59.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:13:59.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:13:59.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:13:59.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:13:59.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:13:59.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:13:59.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:13:59.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:13:59.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:13:59.414 INFO:tasks.workunit.client.0.trial144.stderr:start osd.0 2026-02-19T13:13:59.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:13:59.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:13:59.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:13:59.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:13:59.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:13:59.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:13:59.432 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:59.430+0000 7f22ded9aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:59.434 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:59.430+0000 7f22ded9aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:59.435 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:59.430+0000 7f22ded9aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:13:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:13:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:13:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:13:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:13:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:13:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:13:59.576 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:13:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:13:59.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:13:59.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:13:59.785 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:13:59.782+0000 7f22ded9aa40 -1 Falling back to public interface 2026-02-19T13:14:00.456 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:00.454+0000 7f22ded9aa40 -1 osd.0 20 log_to_monitors true 2026-02-19T13:14:00.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:00.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:00.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:00.735 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:14:00.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:00.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:14:00.892 INFO:tasks.workunit.client.0.trial144.stderr:osd.0 up in weight 1 up_from 23 up_thru 23 down_at 20 last_clean_interval [5,20) [v2:127.0.0.1:6802/2836774729,v1:127.0.0.1:6803/2836774729] [v2:127.0.0.1:6804/2836774729,v1:127.0.0.1:6805/2836774729] exists,up d82e173b-028f-4e5d-85d6-d71c5f9427dd 2026-02-19T13:14:00.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:00.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:00.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:00.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:14:00.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:14:00.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:14:00.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:14:00.893 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:14:00.894 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:14:00.894 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:14:00.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:14:00.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:14:00.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:14:01.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:14:01.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:14:01.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:14:01.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:14:01.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:14:01.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:14:01.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:14:01.170 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:14:01.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:14:01.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:01.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:14:01.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247810 2026-02-19T13:14:01.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247810 2026-02-19T13:14:01.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247810' 2026-02-19T13:14:01.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:01.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:14:01.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672979 2026-02-19T13:14:01.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672979 2026-02-19T13:14:01.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-98784247810 1-42949672979' 2026-02-19T13:14:01.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:01.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-98784247810 2026-02-19T13:14:01.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:01.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:14:01.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-98784247810 2026-02-19T13:14:01.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:01.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=98784247810 2026-02-19T13:14:01.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 98784247810' 2026-02-19T13:14:01.326 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 98784247810 2026-02-19T13:14:01.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:01.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836500 -lt 98784247810 2026-02-19T13:14:01.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:14:02.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:14:02.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:02.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 98784247811 -lt 98784247810 2026-02-19T13:14:02.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:02.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672979 2026-02-19T13:14:02.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:02.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:14:02.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672979 2026-02-19T13:14:02.664 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:02.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672979 2026-02-19T13:14:02.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672979' 2026-02-19T13:14:02.666 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 42949672979 2026-02-19T13:14:02.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:14:02.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672980 -lt 42949672979 2026-02-19T13:14:02.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:14:02.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:02.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:03.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T13:14:03.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:14:03.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:14:03.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:14:03.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:14:03.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:14:03.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:14:03.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:14:03.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T13:14:03.194 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:14:03.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:03.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:03.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T13:14:03.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:14:03.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:14:03.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:240: scrub_and_not_schedule: rados --pool rbd get SOMETHING td/osd-scrub-repair/COPY 2026-02-19T13:14:03.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:241: scrub_and_not_schedule: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-19T13:14:03.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T13:14:03.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:14:03.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:14:03.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:14:03.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:14:03.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:14:03.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:14:03.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:14:03.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:14:03.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:14:03.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:14:03.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:14:03.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:14:03.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:14:03.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:14:03.525 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:14:03.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:14:03.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:14:03.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:14:03.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:14:03.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:14:03.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:14:03.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:14:03.537 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:14:03.537 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:03.537 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:03.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:14:03.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:14:03.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:14:03.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T13:14:03.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T13:14:03.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T13:14:03.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T13:14:03.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:14:03.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:14:03.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:14:03.540 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:14:03.540 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:14:03.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:14:03.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:14:03.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:14:03.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:14:03.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:14:03.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:14:03.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:14:03.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:14:03.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:14:03.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:14:03.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:14:03.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:14:03.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:14:03.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:14:03.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:14:03.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:14:03.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:14:03.553 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:14:03.553 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:03.553 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:03.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:14:03.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:14:03.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:14:03.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T13:14:03.557 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:14:03.557 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:03.557 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:03.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T13:14:03.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:14:03.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:14:03.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:14:03.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T13:14:03.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_unfound_erasure_coded_appends td/osd-scrub-repair 2026-02-19T13:14:03.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:962: TEST_unfound_erasure_coded_appends: unfound_erasure_coded td/osd-scrub-repair false 2026-02-19T13:14:03.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:911: unfound_erasure_coded: local dir=td/osd-scrub-repair 2026-02-19T13:14:03.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:912: unfound_erasure_coded: local allow_overwrites=false 2026-02-19T13:14:03.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:913: unfound_erasure_coded: local poolname=ecpool 2026-02-19T13:14:03.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:914: unfound_erasure_coded: local payload=ABCDEF 2026-02-19T13:14:03.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:916: unfound_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-19T13:14:03.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T13:14:03.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:14:03.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:14:03.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:14:03.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T13:14:03.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T13:14:03.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:14:03.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:03.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:03.583 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:03.583 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:03.583 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:03.584 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:03.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:14:03.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:14:03.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:14:03.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:14:03.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:14:03.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:14:03.613 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:14:03.613 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:14:03.613 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:14:03.613 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:14:03.614 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:14:03.614 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:03.614 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:03.614 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:14:03.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:14:03.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T13:14:03.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:14:03.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:14:03.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:14:03.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:14:03.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:14:03.670 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:14:03.671 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:14:03.671 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:14:03.671 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:14:03.671 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:03.672 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:03.672 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:14:03.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:14:03.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T13:14:03.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:917: unfound_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-19T13:14:03.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T13:14:03.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:14:03.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:14:03.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:14:03.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T13:14:03.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:14:03.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:14:03.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:03.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:03.824 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:03.824 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:03.824 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:03.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:03.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:14:03.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:14:03.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:918: unfound_erasure_coded: seq 0 3 2026-02-19T13:14:03.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:918: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-19T13:14:03.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:919: unfound_erasure_coded: run_osd td/osd-scrub-repair 0 2026-02-19T13:14:03.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:14:03.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:03.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:14:03.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:03.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:14:03.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:14:03.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:03.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:03.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:03.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:14:03.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:14:03.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:03.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:03.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:14:03.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:03.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:03.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:03.850 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:03.850 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:03.850 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:03.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:03.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:03.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:03.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:03.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:03.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:14:03.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:14:03.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:03.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:03.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:03.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:03.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:03.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:03.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:14:03.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:03.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4e8d07ca-8569-4da0-96ab-56474932b10f 2026-02-19T13:14:03.852 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 4e8d07ca-8569-4da0-96ab-56474932b10f 2026-02-19T13:14:03.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 4e8d07ca-8569-4da0-96ab-56474932b10f' 2026-02-19T13:14:03.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:03.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCbDJdp1AZ6MxAAkDfhHCJ49XKNsOe3Xab3YQ== 2026-02-19T13:14:03.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCbDJdp1AZ6MxAAkDfhHCJ49XKNsOe3Xab3YQ=="}' 2026-02-19T13:14:03.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4e8d07ca-8569-4da0-96ab-56474932b10f -i td/osd-scrub-repair/0/new.json 2026-02-19T13:14:03.952 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:14:03.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T13:14:03.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCbDJdp1AZ6MxAAkDfhHCJ49XKNsOe3Xab3YQ== --osd-uuid 4e8d07ca-8569-4da0-96ab-56474932b10f 2026-02-19T13:14:03.977 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:03.974+0000 7ffa5da30a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:03.979 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:03.978+0000 7ffa5da30a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:03.980 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:03.978+0000 7ffa5da30a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:03.980 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:03.978+0000 7ffa5da30a40 -1 bdev(0x55fee1ec6c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:03.981 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:03.978+0000 7ffa5da30a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T13:14:04.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T13:14:04.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:04.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:14:04.727 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:14:04.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:04.930 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:14:04.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:14:04.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:04.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:04.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:04.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:04.955 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:04.954+0000 7f55eed29a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:04.957 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:04.954+0000 7f55eed29a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:04.958 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:04.954+0000 7f55eed29a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:05.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:14:05.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:05.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:14:05.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:05.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:05.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:05.088 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:14:05.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:05.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:05.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:14:05.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:05.381 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:05.378+0000 7f55eed29a40 -1 Falling back to public interface 2026-02-19T13:14:05.975 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:05.974+0000 7f55eed29a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:14:06.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:06.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:06.246 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:14:06.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:06.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:06.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:14:06.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:07.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:07.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:07.414 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:14:07.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:14:07.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:07.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:14:07.570 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/983903037,v1:127.0.0.1:6803/983903037] [v2:127.0.0.1:6804/983903037,v1:127.0.0.1:6805/983903037] exists,up 4e8d07ca-8569-4da0-96ab-56474932b10f 2026-02-19T13:14:07.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:07.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:07.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:07.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:918: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-19T13:14:07.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:919: unfound_erasure_coded: run_osd td/osd-scrub-repair 1 2026-02-19T13:14:07.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:14:07.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:07.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:14:07.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:07.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:14:07.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:14:07.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:07.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:07.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:07.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:14:07.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:14:07.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:07.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:07.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:14:07.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:07.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:07.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:07.576 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:07.577 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:07.577 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:07.577 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:07.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:07.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:07.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:07.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:07.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:14:07.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:14:07.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:07.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:07.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:07.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:07.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:07.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:07.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:14:07.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:07.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7bef4be6-7d19-4cc5-b0e4-5842f2dad712 2026-02-19T13:14:07.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 7bef4be6-7d19-4cc5-b0e4-5842f2dad712' 2026-02-19T13:14:07.579 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 7bef4be6-7d19-4cc5-b0e4-5842f2dad712 2026-02-19T13:14:07.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:07.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCfDJdp/D4eIxAAdnZriw5sW0Hp4F97t+Xlng== 2026-02-19T13:14:07.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCfDJdp/D4eIxAAdnZriw5sW0Hp4F97t+Xlng=="}' 2026-02-19T13:14:07.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7bef4be6-7d19-4cc5-b0e4-5842f2dad712 -i td/osd-scrub-repair/1/new.json 2026-02-19T13:14:07.753 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:14:07.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T13:14:07.764 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCfDJdp/D4eIxAAdnZriw5sW0Hp4F97t+Xlng== --osd-uuid 7bef4be6-7d19-4cc5-b0e4-5842f2dad712 2026-02-19T13:14:07.781 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:07.778+0000 7efce5b17a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:07.783 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:07.778+0000 7efce5b17a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:07.784 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:07.782+0000 7efce5b17a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:07.784 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:07.782+0000 7efce5b17a40 -1 bdev(0x5587af3bcc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:07.784 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:07.782+0000 7efce5b17a40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T13:14:08.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T13:14:08.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:08.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:14:08.420 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:14:08.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:08.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:14:08.622 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:14:08.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:08.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:08.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:08.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:08.646 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:08.642+0000 7fec6eb5ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:08.648 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:08.646+0000 7fec6eb5ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:08.649 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:08.646+0000 7fec6eb5ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:08.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:14:08.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:08.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:14:08.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:08.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:08.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:08.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:08.779 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:14:08.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:08.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:14:08.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:09.073 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:09.070+0000 7fec6eb5ca40 -1 Falling back to public interface 2026-02-19T13:14:09.658 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:09.654+0000 7fec6eb5ca40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:14:09.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:09.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:09.939 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:14:09.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:09.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:09.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:14:10.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:11.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:11.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:11.106 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:14:11.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:14:11.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:11.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:14:11.267 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2421868535,v1:127.0.0.1:6811/2421868535] [v2:127.0.0.1:6812/2421868535,v1:127.0.0.1:6813/2421868535] exists,up 7bef4be6-7d19-4cc5-b0e4-5842f2dad712 2026-02-19T13:14:11.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:11.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:11.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:11.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:918: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-19T13:14:11.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:919: unfound_erasure_coded: run_osd td/osd-scrub-repair 2 2026-02-19T13:14:11.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:14:11.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:11.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:14:11.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:11.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:14:11.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:14:11.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:11.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:11.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:11.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T13:14:11.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T13:14:11.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:11.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:11.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:14:11.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:11.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:11.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:11.276 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:11.276 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:11.276 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:11.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:11.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:11.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:11.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:11.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:11.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:14:11.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:14:11.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:11.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:11.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:11.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:11.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:11.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:11.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T13:14:11.277 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:11.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b3ee9df4-f72f-4d90-844e-ed0facf7e497 2026-02-19T13:14:11.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 b3ee9df4-f72f-4d90-844e-ed0facf7e497' 2026-02-19T13:14:11.278 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 b3ee9df4-f72f-4d90-844e-ed0facf7e497 2026-02-19T13:14:11.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:11.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCjDJdp0nYIERAARKGiK4DLyd7fXSOHnu+/6g== 2026-02-19T13:14:11.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCjDJdp0nYIERAARKGiK4DLyd7fXSOHnu+/6g=="}' 2026-02-19T13:14:11.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b3ee9df4-f72f-4d90-844e-ed0facf7e497 -i td/osd-scrub-repair/2/new.json 2026-02-19T13:14:11.451 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:14:11.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T13:14:11.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCjDJdp0nYIERAARKGiK4DLyd7fXSOHnu+/6g== --osd-uuid b3ee9df4-f72f-4d90-844e-ed0facf7e497 2026-02-19T13:14:11.478 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:11.474+0000 7fcc5af54a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:11.479 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:11.478+0000 7fcc5af54a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:11.480 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:11.478+0000 7fcc5af54a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:11.481 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:11.478+0000 7fcc5af54a40 -1 bdev(0x55f102358c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:11.481 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:11.478+0000 7fcc5af54a40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T13:14:12.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T13:14:12.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:12.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:14:12.104 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T13:14:12.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:12.301 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T13:14:12.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:14:12.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:12.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:12.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:12.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:12.318 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:12.314+0000 7fae73263a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:12.319 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:12.318+0000 7fae73263a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:12.320 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:12.318+0000 7fae73263a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:14:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:12.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:14:12.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:12.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:12.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:12.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:12.456 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:14:12.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:12.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:14:12.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:12.741 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:12.738+0000 7fae73263a40 -1 Falling back to public interface 2026-02-19T13:14:13.301 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:13.298+0000 7fae73263a40 -1 osd.2 0 log_to_monitors true 2026-02-19T13:14:13.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:13.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:13.617 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:14:13.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:13.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:13.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:14:13.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:14.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:14.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:14.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:14:14.777 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:14:14.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:14.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:14:14.937 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/437184,v1:127.0.0.1:6819/437184] [v2:127.0.0.1:6820/437184,v1:127.0.0.1:6821/437184] exists,up b3ee9df4-f72f-4d90-844e-ed0facf7e497 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:918: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:919: unfound_erasure_coded: run_osd td/osd-scrub-repair 3 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:14.938 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:14.939 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:14.939 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:14.939 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:14.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:14.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:14.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:14.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:14.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:14:14.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:14:14.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:14.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:14.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:14.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:14.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:14.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:14.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T13:14:14.942 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:14.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e3712acc-9635-41c9-ba6d-f9a55423ef22 2026-02-19T13:14:14.943 INFO:tasks.workunit.client.0.trial144.stdout:add osd3 e3712acc-9635-41c9-ba6d-f9a55423ef22 2026-02-19T13:14:14.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 e3712acc-9635-41c9-ba6d-f9a55423ef22' 2026-02-19T13:14:14.943 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:14.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCmDJdpYxruOBAADjya1b4OHeYd0D4EdVoybw== 2026-02-19T13:14:14.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCmDJdpYxruOBAADjya1b4OHeYd0D4EdVoybw=="}' 2026-02-19T13:14:14.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e3712acc-9635-41c9-ba6d-f9a55423ef22 -i td/osd-scrub-repair/3/new.json 2026-02-19T13:14:15.117 INFO:tasks.workunit.client.0.trial144.stdout:3 2026-02-19T13:14:15.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-19T13:14:15.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCmDJdpYxruOBAADjya1b4OHeYd0D4EdVoybw== --osd-uuid e3712acc-9635-41c9-ba6d-f9a55423ef22 2026-02-19T13:14:15.143 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:15.142+0000 7eff475d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:15.145 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:15.142+0000 7eff475d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:15.146 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:15.142+0000 7eff475d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:15.146 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:15.142+0000 7eff475d7a40 -1 bdev(0x5628a90b8c00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:15.146 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:15.142+0000 7eff475d7a40 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-19T13:14:15.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-19T13:14:15.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:15.813 INFO:tasks.workunit.client.0.trial144.stdout:adding osd3 key to auth repository 2026-02-19T13:14:15.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:14:15.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:16.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:14:16.011 INFO:tasks.workunit.client.0.trial144.stdout:start osd.3 2026-02-19T13:14:16.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:16.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:16.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:16.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:16.035 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:16.034+0000 7f68c1fe0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:16.037 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:16.034+0000 7f68c1fe0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:16.038 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:16.034+0000 7f68c1fe0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:16.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:14:16.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:16.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:14:16.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:16.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:16.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:16.168 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:14:16.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:16.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:16.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:14:16.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:16.425 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:16.422+0000 7f68c1fe0a40 -1 Falling back to public interface 2026-02-19T13:14:16.997 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:16.994+0000 7f68c1fe0a40 -1 osd.3 0 log_to_monitors true 2026-02-19T13:14:17.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:17.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:17.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:17.334 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:14:17.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:17.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:14:17.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:18.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:18.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:18.497 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:14:18.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:14:18.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:18.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:14:18.658 INFO:tasks.workunit.client.0.trial144.stdout:osd.3 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2499010757,v1:127.0.0.1:6827/2499010757] [v2:127.0.0.1:6828/2499010757,v1:127.0.0.1:6829/2499010757] exists,up e3712acc-9635-41c9-ba6d-f9a55423ef22 2026-02-19T13:14:18.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:18.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:18.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:18.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: unfound_erasure_coded: create_ec_pool ecpool false k=2 m=2 2026-02-19T13:14:18.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: create_ec_pool: local pool_name=ecpool 2026-02-19T13:14:18.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: create_ec_pool: shift 2026-02-19T13:14:18.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: create_ec_pool: local allow_overwrites=false 2026-02-19T13:14:18.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2525: create_ec_pool: shift 2026-02-19T13:14:18.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=2 2026-02-19T13:14:18.862 INFO:tasks.workunit.client.0.trial144.stderr:load: isa 2026-02-19T13:14:18.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-19T13:14:18.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-19T13:14:19.168 INFO:tasks.workunit.client.0.trial144.stderr:pool 'ecpool' created 2026-02-19T13:14:19.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:14:20.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: create_ec_pool: '[' false = true ']' 2026-02-19T13:14:20.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2535: create_ec_pool: wait_for_clean 2026-02-19T13:14:20.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:14:20.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:14:20.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:14:20.183 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:14:20.183 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:14:20.183 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:14:20.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:14:20.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:14:20.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:14:20.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:14:20.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:14:20.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:14:20.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:14:20.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:14:20.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:14:20.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:14:20.459 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:14:20.459 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T13:14:20.459 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T13:14:20.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:14:20.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:20.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:14:20.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-19T13:14:20.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-19T13:14:20.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-19T13:14:20.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:20.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:14:20.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-19T13:14:20.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-19T13:14:20.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-38654705667' 2026-02-19T13:14:20.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:20.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:14:20.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574851 2026-02-19T13:14:20.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574851 2026-02-19T13:14:20.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-38654705667 2-55834574851' 2026-02-19T13:14:20.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:20.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:14:20.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444034 2026-02-19T13:14:20.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444034 2026-02-19T13:14:20.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-38654705667 2-55834574851 3-73014444034' 2026-02-19T13:14:20.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:20.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-19T13:14:20.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:20.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:14:20.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-19T13:14:20.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:20.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-19T13:14:20.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-19T13:14:20.751 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836484 2026-02-19T13:14:20.752 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:20.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-19T13:14:20.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:14:21.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:14:21.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:22.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-19T13:14:22.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:14:23.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:14:23.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:23.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836484 2026-02-19T13:14:23.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:23.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-19T13:14:23.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:23.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:14:23.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705667 2026-02-19T13:14:23.236 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:23.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705667 2026-02-19T13:14:23.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-19T13:14:23.238 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705667 2026-02-19T13:14:23.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:14:23.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705668 -lt 38654705667 2026-02-19T13:14:23.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:23.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574851 2026-02-19T13:14:23.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:23.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:14:23.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574851 2026-02-19T13:14:23.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:23.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574851 2026-02-19T13:14:23.403 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574851 2026-02-19T13:14:23.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-19T13:14:23.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:14:23.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-19T13:14:23.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:23.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-73014444034 2026-02-19T13:14:23.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:23.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:14:23.562 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-73014444034 2026-02-19T13:14:23.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:23.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444034 2026-02-19T13:14:23.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 73014444034' 2026-02-19T13:14:23.565 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.3 seq 73014444034 2026-02-19T13:14:23.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:14:23.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444034 -lt 73014444034 2026-02-19T13:14:23.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:14:23.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:23.729 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:23.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:14:23.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:14:23.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:14:23.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:14:23.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:14:23.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:14:23.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:14:23.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:14:24.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:14:24.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:14:24.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:24.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:24.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:14:24.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:14:24.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:14:24.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2536: create_ec_pool: return 0 2026-02-19T13:14:24.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:924: unfound_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-19T13:14:24.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:14:24.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T13:14:24.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-19T13:14:24.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:14:24.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:14:24.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:14:24.510 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:14:24.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:14:24.712 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:14:24.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:14:24.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:14:24.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-19T13:14:24.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:926: unfound_erasure_coded: get_primary ecpool SOMETHING 2026-02-19T13:14:24.752 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=ecpool 2026-02-19T13:14:24.752 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T13:14:24.752 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:14:24.753 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-19T13:14:24.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:926: unfound_erasure_coded: local primary=1 2026-02-19T13:14:24.914 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:927: unfound_erasure_coded: get_osds ecpool SOMETHING 2026-02-19T13:14:24.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T13:14:24.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:927: unfound_erasure_coded: sed -e s/1// 2026-02-19T13:14:24.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:14:24.915 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-19T13:14:24.916 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:14:25.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-19T13:14:25.075 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:14:25.076 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T13:14:25.076 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T13:14:25.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2 3 2026-02-19T13:14:25.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:927: unfound_erasure_coded: osds=('0' '2' '3') 2026-02-19T13:14:25.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:927: unfound_erasure_coded: local -a osds 2026-02-19T13:14:25.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:928: unfound_erasure_coded: local not_primary_first=0 2026-02-19T13:14:25.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:929: unfound_erasure_coded: local not_primary_second=2 2026-02-19T13:14:25.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:930: unfound_erasure_coded: local not_primary_third=3 2026-02-19T13:14:25.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:935: unfound_erasure_coded: pids= 2026-02-19T13:14:25.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:936: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T13:14:25.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T13:14:25.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T13:14:25.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 441049"' 2026-02-19T13:14:25.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 441049' 2026-02-19T13:14:25.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:937: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T13:14:25.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T13:14:25.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T13:14:25.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 441050"' 2026-02-19T13:14:25.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 441050' 2026-02-19T13:14:25.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:938: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T13:14:25.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T13:14:25.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T13:14:25.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T13:14:25.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/441052: /' 2026-02-19T13:14:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 441053"' 2026-02-19T13:14:25.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 441053' 2026-02-19T13:14:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T13:14:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: unfound_erasure_coded: wait_background pids 2026-02-19T13:14:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 441049 441050 441053' 2026-02-19T13:14:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T13:14:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T13:14:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 441049 2026-02-19T13:14:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/441055: /' 2026-02-19T13:14:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T13:14:25.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/441059: /' 2026-02-19T13:14:25.708 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:14:25.708 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:14:25.708 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T13:14:25.708 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:14:25.708 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T13:14:25.708 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:14:25.708 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:14:25.709 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T13:14:25.709 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:14:25.709 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T13:14:25.709 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:14:25.709 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:14:25.709 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:14:25.709 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:14:25.709 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:14:25.709 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:14:25.709 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T13:14:25.710 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:14:25.710 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:14:25.710 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T13:14:25.710 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:14:25.710 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T13:14:25.710 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-19T13:14:25.710 INFO:tasks.workunit.client.0.trial144.stderr:441059: remove 3#1:eb822e21:::SOMETHING:head# 2026-02-19T13:14:25.710 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T13:14:25.710 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:14:25.710 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:14:25.710 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T13:14:25.711 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:14:25.711 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T13:14:25.711 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:14:25.711 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:25.712 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:25.712 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:25.713 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T13:14:25.713 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T13:14:25.713 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:25.713 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:14:25.713 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:14:25.713 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:14:25.713 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:25.713 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:25.713 INFO:tasks.workunit.client.0.trial144.stderr:441059: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:25.713 INFO:tasks.workunit.client.0.trial144.stderr:441059: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:25.714 INFO:tasks.workunit.client.0.trial144.stderr:441059: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:25.714 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:25.714 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:25.714 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:25.714 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:14:25.714 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:14:25.714 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:25.714 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:25.714 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:25.714 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:25.714 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:14:25.715 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:14:25.715 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T13:14:25.715 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T13:14:25.715 INFO:tasks.workunit.client.0.trial144.stderr:441059: start osd.3 2026-02-19T13:14:25.715 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:25.724 INFO:tasks.workunit.client.0.trial144.stderr:4441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:14:25.724 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:14:25.724 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:14:25.724 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:14:25.724 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T13:14:25.724 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:14:25.724 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:14:25.725 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:14:25.725 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:14:25.725 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:14:25.725 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:14:25.725 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:14:25.725 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:14:25.725 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:14:25.725 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:14:25.725 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:14:25.725 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T13:14:25.725 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:14:25.726 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:14:25.726 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:14:25.726 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:14:25.726 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:14:25.726 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-19T13:14:25.726 INFO:tasks.workunit.client.0.trial144.stderr:441052: remove 1#1:eb822e21:::SOMETHING:head# 2026-02-19T13:14:25.726 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:14:25.726 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:14:25.726 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:14:25.726 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:14:25.726 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:14:25.727 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:14:25.727 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:14:25.727 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:25.728 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:25.728 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:25.728 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:14:25.729 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:14:25.729 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:25.729 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:14:25.729 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:14:25.729 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:14:25.729 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:25.729 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:25.729 INFO:tasks.workunit.client.0.trial144.stderr:441052: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:25.729 INFO:tasks.workunit.client.0.trial144.stderr:441052: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:25.729 INFO:tasks.workunit.client.0.trial144.stderr:441052: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:25.729 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:25.730 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:25.730 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:25.730 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:14:25.730 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:14:25.730 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:25.730 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:25.730 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:25.730 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:25.730 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:14:25.730 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:14:25.730 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:14:25.731 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:14:25.731 INFO:tasks.workunit.client.0.trial144.stderr:441052: start osd.0 2026-02-19T13:14:25.731 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:25.779 INFO:tasks.workunit.client.0.trial144.stderr:4441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:14:25.779 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:14:25.780 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T13:14:25.780 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:14:25.780 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T13:14:25.780 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:14:25.780 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:14:25.780 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T13:14:25.780 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:14:25.780 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-19T13:14:25.780 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:14:25.780 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:14:25.781 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:14:25.781 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:14:25.781 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:14:25.781 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:14:25.781 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T13:14:25.781 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:14:25.781 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:14:25.781 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T13:14:25.781 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:14:25.781 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:14:25.781 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING remove 2026-02-19T13:14:25.781 INFO:tasks.workunit.client.0.trial144.stderr:441055: remove 2#1:eb822e21:::SOMETHING:head# 2026-02-19T13:14:25.781 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-19T13:14:25.782 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:14:25.782 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:14:25.782 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:14:25.782 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:14:25.782 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:14:25.782 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:14:25.782 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:25.783 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:25.784 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:25.784 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T13:14:25.784 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T13:14:25.784 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:25.784 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:14:25.784 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:14:25.784 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:14:25.784 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:25.784 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:25.784 INFO:tasks.workunit.client.0.trial144.stderr:441055: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:25.784 INFO:tasks.workunit.client.0.trial144.stderr:441055: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:25.785 INFO:tasks.workunit.client.0.trial144.stderr:441055: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:25.785 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:25.785 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:25.785 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:25.785 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:14:25.785 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:14:25.785 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:25.785 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:25.785 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:25.785 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:25.785 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:14:25.785 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:14:25.786 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T13:14:25.786 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:14:25.786 INFO:tasks.workunit.client.0.trial144.stderr:441055: start osd.2 2026-02-19T13:14:25.786 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:27.196 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T13:14:27.197 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T13:14:27.197 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:14:27.197 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:14:27.197 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:14:27.197 INFO:tasks.workunit.client.0.trial144.stderr:441059: 2026-02-19T13:14:25.726+0000 7fe1fd766a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:27.197 INFO:tasks.workunit.client.0.trial144.stderr:441059: 2026-02-19T13:14:25.726+0000 7fe1fd766a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:27.197 INFO:tasks.workunit.client.0.trial144.stderr:441059: 2026-02-19T13:14:25.726+0000 7fe1fd766a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:27.197 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T13:14:27.198 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:27.198 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:14:27.198 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:27.198 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:27.198 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:27.198 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:27.198 INFO:tasks.workunit.client.0.trial144.stderr:441059: 0 2026-02-19T13:14:27.198 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:27.198 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:14:27.198 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:27.198 INFO:tasks.workunit.client.0.trial144.stderr:441059: 2026-02-19T13:14:26.218+0000 7fe1fd766a40 -1 Falling back to public interface 2026-02-19T13:14:27.198 INFO:tasks.workunit.client.0.trial144.stderr:441059: 2026-02-19T13:14:26.962+0000 7fe1fd766a40 -1 osd.3 24 log_to_monitors true 2026-02-19T13:14:27.199 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:27.199 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:27.199 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:27.199 INFO:tasks.workunit.client.0.trial144.stderr:441059: 1 2026-02-19T13:14:27.199 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:27.199 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:14:27.199 INFO:tasks.workunit.client.0.trial144.stderr:441059: osd.3 up in weight 1 up_from 28 up_thru 0 down_at 25 last_clean_interval [17,24) [v2:127.0.0.1:6802/752624420,v1:127.0.0.1:6803/752624420] [v2:127.0.0.1:6804/752624420,v1:127.0.0.1:6805/752624420] exists,up e3712acc-9635-41c9-ba6d-f9a55423ef22 2026-02-19T13:14:27.199 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:27.199 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:27.199 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:27.200 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:14:27.200 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:14:27.200 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:14:27.200 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:14:27.200 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:14:27.200 INFO:tasks.workunit.client.0.trial144.stderr:441059: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:14:27.230 INFO:tasks.workunit.client.0.trial144.stderr:441059: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_ti41052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:14:27.230 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:14:27.230 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:14:27.230 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:14:27.230 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:14:27.230 INFO:tasks.workunit.client.0.trial144.stderr:441052: 2026-02-19T13:14:25.742+0000 7f9a99f43a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:27.230 INFO:tasks.workunit.client.0.trial144.stderr:441052: 2026-02-19T13:14:25.746+0000 7f9a99f43a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:27.230 INFO:tasks.workunit.client.0.trial144.stderr:441052: 2026-02-19T13:14:25.746+0000 7f9a99f43a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:27.230 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:14:27.230 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:27.230 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:14:27.230 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:27.231 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:27.231 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:27.231 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:27.231 INFO:tasks.workunit.client.0.trial144.stderr:441052: 0 2026-02-19T13:14:27.231 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:27.231 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:14:27.231 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:27.231 INFO:tasks.workunit.client.0.trial144.stderr:441052: 2026-02-19T13:14:26.250+0000 7f9a99f43a40 -1 Falling back to public interface 2026-02-19T13:14:27.231 INFO:tasks.workunit.client.0.trial144.stderr:441052: 2026-02-19T13:14:26.994+0000 7f9a99f43a40 -1 osd.0 24 log_to_monitors true 2026-02-19T13:14:27.231 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:27.231 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:27.232 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:27.232 INFO:tasks.workunit.client.0.trial144.stderr:441052: 1 2026-02-19T13:14:27.232 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:27.232 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:14:27.232 INFO:tasks.workunit.client.0.trial144.stderr:441052: osd.0 up in weight 1 up_from 29 up_thru 0 down_at 25 last_clean_interval [5,24) [v2:127.0.0.1:6818/2428152974,v1:127.0.0.1:6819/2428152974] [v2:127.0.0.1:6820/2428152974,v1:127.0.0.1:6821/2428152974] exists,up 4e8d07ca-8569-4da0-96ab-56474932b10f 2026-02-19T13:14:27.232 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:27.232 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:27.232 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:27.232 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:14:27.232 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:14:27.232 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:14:27.232 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:14:27.233 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:14:27.233 INFO:tasks.workunit.client.0.trial144.stderr:441052: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:14:27.286 INFO:tasks.workunit.client.0.trial144.stderr:441052: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get41055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-19T13:14:27.286 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:14:27.286 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:14:27.286 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:14:27.286 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:14:27.286 INFO:tasks.workunit.client.0.trial144.stderr:441055: 2026-02-19T13:14:25.794+0000 7f8fa0164a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:27.286 INFO:tasks.workunit.client.0.trial144.stderr:441055: 2026-02-19T13:14:25.794+0000 7f8fa0164a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:27.287 INFO:tasks.workunit.client.0.trial144.stderr:441055: 2026-02-19T13:14:25.798+0000 7f8fa0164a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:27.287 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:14:27.287 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:27.287 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:14:27.287 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:27.287 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:27.287 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:27.287 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:27.287 INFO:tasks.workunit.client.0.trial144.stderr:441055: 0 2026-02-19T13:14:27.287 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:27.288 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:14:27.288 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:27.288 INFO:tasks.workunit.client.0.trial144.stderr:441055: 2026-02-19T13:14:26.270+0000 7f8fa0164a40 -1 Falling back to public interface 2026-02-19T13:14:27.288 INFO:tasks.workunit.client.0.trial144.stderr:441055: 2026-02-19T13:14:27.018+0000 7f8fa0164a40 -1 osd.2 24 log_to_monitors true 2026-02-19T13:14:27.288 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:27.288 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:27.288 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:27.288 INFO:tasks.workunit.client.0.trial144.stderr:441055: 1 2026-02-19T13:14:27.288 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:27.288 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:14:27.289 INFO:tasks.workunit.client.0.trial144.stderr:441055: osd.2 up in weight 1 up_from 29 up_thru 0 down_at 25 last_clean_interval [13,24) [v2:127.0.0.1:6826/4273107843,v1:127.0.0.1:6827/4273107843] [v2:127.0.0.1:6828/4273107843,v1:127.0.0.1:6829/4273107843] exists,up b3ee9df4-f72f-4d90-844e-ed0facf7e497 2026-02-19T13:14:27.289 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:27.289 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:27.289 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:27.289 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:14:27.289 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:14:27.289 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:14:27.289 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:14:27.289 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:14:27.289 INFO:tasks.workunit.client.0.trial144.stderr:441055: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:14:27.773 INFO:tasks.workunit.client.0.trial144.stderr:441055: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: gemeout_delays: echo true 2026-02-19T13:14:27.773 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:14:27.773 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:14:27.773 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:14:27.773 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:14:27.773 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:14:27.773 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: 1 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: 2 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: 3' 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051586 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051586 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-124554051586' 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705670 2026-02-19T13:14:27.774 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705670 2026-02-19T13:14:27.775 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-124554051586 1-38654705670' 2026-02-19T13:14:27.775 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:27.775 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:14:27.775 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051586 2026-02-19T13:14:27.775 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051586 2026-02-19T13:14:27.775 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-124554051586 1-38654705670 2-124554051586' 2026-02-19T13:14:27.775 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:27.775 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:14:27.775 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084290 2026-02-19T13:14:27.775 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084290 2026-02-19T13:14:27.775 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-124554051586 1-38654705670 2-124554051586 3-120259084290' 2026-02-19T13:14:27.775 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:27.775 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-124554051586 2026-02-19T13:14:27.811 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubu_timeout_delays: echo true 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: 1 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: 2 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: 3' 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:27.812 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051587 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051587 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-124554051587' 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705671 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705671 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-124554051587 1-38654705671' 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051587 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051587 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-124554051587 1-38654705671 2-124554051587' 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084291 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084291 2026-02-19T13:14:27.813 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-124554051587 1-38654705671 2-124554051587 3-120259084291' 2026-02-19T13:14:27.814 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:27.814 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-124554051587 2026-02-19T13:14:27.858 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/t_timeout_delays: echo true 2026-02-19T13:14:27.858 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:14:27.858 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:14:27.858 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:14:27.858 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:14:27.858 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:14:27.858 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:14:27.858 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:14:27.858 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:14:27.858 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:14:27.859 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:14:27.859 INFO:tasks.workunit.client.0.trial144.stderr:441055: 1 2026-02-19T13:14:27.859 INFO:tasks.workunit.client.0.trial144.stderr:441055: 2 2026-02-19T13:14:27.859 INFO:tasks.workunit.client.0.trial144.stderr:441055: 3' 2026-02-19T13:14:27.859 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:14:27.859 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:27.859 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:14:27.859 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051588 2026-02-19T13:14:27.859 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051588 2026-02-19T13:14:27.859 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-124554051588' 2026-02-19T13:14:27.859 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:27.859 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:14:27.859 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705672 2026-02-19T13:14:27.859 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705672 2026-02-19T13:14:27.859 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-124554051588 1-38654705672' 2026-02-19T13:14:27.860 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:27.860 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:14:27.860 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051588 2026-02-19T13:14:27.860 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051588 2026-02-19T13:14:27.860 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-124554051588 1-38654705672 2-124554051588' 2026-02-19T13:14:27.860 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:27.860 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:14:27.860 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084292 2026-02-19T13:14:27.860 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084292 2026-02-19T13:14:27.860 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-124554051588 1-38654705672 2-124554051588 3-120259084292' 2026-02-19T13:14:27.860 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:27.860 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-124554051588 2026-02-19T13:14:29.471 INFO:tasks.workunit.client.0.trial144.stderr:441055: //homentu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:29.471 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:14:29.471 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-124554051586 2026-02-19T13:14:29.471 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:29.471 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051586 2026-02-19T13:14:29.471 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 124554051586' 2026-02-19T13:14:29.471 INFO:tasks.workunit.client.0.trial144.stderr:441059: waiting osd.0 seq 124554051586 2026-02-19T13:14:29.471 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:29.471 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 124554051586 2026-02-19T13:14:29.471 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:14:29.471 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:14:29.471 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:29.471 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051588 -lt 124554051586 2026-02-19T13:14:29.471 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:29.471 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705670 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705670 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705670 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: waiting osd.1 seq 38654705670 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705672 -lt 38654705670 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-124554051586 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-124554051586 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051586 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 124554051586' 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: waiting osd.2 seq 124554051586 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051588 -lt 124554051586 2026-02-19T13:14:29.472 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:29.475 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:22ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:29.475 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:14:29.475 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-124554051587 2026-02-19T13:14:29.475 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:29.475 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051587 2026-02-19T13:14:29.475 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 124554051587' 2026-02-19T13:14:29.475 INFO:tasks.workunit.client.0.trial144.stderr:441052: waiting osd.0 seq 124554051587 2026-02-19T13:14:29.476 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:29.476 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 124554051587 2026-02-19T13:14:29.476 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:14:29.476 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:14:29.476 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:29.476 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051588 -lt 124554051587 2026-02-19T13:14:29.476 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:29.476 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705671 2026-02-19T13:14:29.476 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:29.476 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:14:29.476 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705671 2026-02-19T13:14:29.477 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:29.477 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705671 2026-02-19T13:14:29.477 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-19T13:14:29.477 INFO:tasks.workunit.client.0.trial144.stderr:441052: waiting osd.1 seq 38654705671 2026-02-19T13:14:29.477 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:14:29.477 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705672 -lt 38654705671 2026-02-19T13:14:29.477 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:29.477 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-124554051587 2026-02-19T13:14:29.477 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:29.477 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:14:29.477 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-124554051587 2026-02-19T13:14:29.477 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:29.478 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051587 2026-02-19T13:14:29.478 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 124554051587' 2026-02-19T13:14:29.478 INFO:tasks.workunit.client.0.trial144.stderr:441052: waiting osd.2 seq 124554051587 2026-02-19T13:14:29.478 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:14:29.478 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051588 -lt 124554051587 2026-02-19T13:14:29.478 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:29.540 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:29.540 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:14:29.540 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-124554051588 2026-02-19T13:14:29.540 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051588 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 124554051588' 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: waiting osd.0 seq 124554051588 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 124554051588 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051588 -lt 124554051588 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705672 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705672 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705672 2026-02-19T13:14:29.541 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705672' 2026-02-19T13:14:29.542 INFO:tasks.workunit.client.0.trial144.stderr:441055: waiting osd.1 seq 38654705672 2026-02-19T13:14:29.542 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:14:29.542 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705672 -lt 38654705672 2026-02-19T13:14:29.542 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:29.542 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-124554051588 2026-02-19T13:14:29.542 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:29.542 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:14:29.542 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-124554051588 2026-02-19T13:14:29.542 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:29.542 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051588 2026-02-19T13:14:29.542 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 124554051588' 2026-02-19T13:14:29.542 INFO:tasks.workunit.client.0.trial144.stderr:441055: waiting osd.2 seq 124554051588 2026-02-19T13:14:29.542 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:14:29.542 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051588 -lt 124554051588 2026-02-19T13:14:29.542 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:30.190 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.s67: flush_pg_stats: echo 3-120259084290 2026-02-19T13:14:30.190 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:30.190 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:14:30.190 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-120259084290 2026-02-19T13:14:30.191 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:30.191 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=120259084290 2026-02-19T13:14:30.191 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 120259084290' 2026-02-19T13:14:30.191 INFO:tasks.workunit.client.0.trial144.stderr:441059: waiting osd.3 seq 120259084290 2026-02-19T13:14:30.191 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:14:30.191 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084292 -lt 120259084290 2026-02-19T13:14:30.191 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:14:30.191 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:30.191 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:30.191 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:14:30.191 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:14:30.191 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:14:30.192 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:14:30.192 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:14:30.192 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:14:30.192 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:14:30.192 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:14:30.192 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:14:30.192 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:14:30.192 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:30.192 INFO:tasks.workunit.client.0.trial144.stderr:441059: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:30.192 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:14:30.192 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:14:30.192 INFO:tasks.workunit.client.0.trial144.stderr:441059: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:14:30.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T13:14:30.219 INFO:tasks.workunit.client.0.trial144.stderr::2267: flush_pg_stats: echo 3-120259084291 2026-02-19T13:14:30.220 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:30.220 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:14:30.220 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-120259084291 2026-02-19T13:14:30.220 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:30.220 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=120259084291 2026-02-19T13:14:30.220 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 120259084291' 2026-02-19T13:14:30.220 INFO:tasks.workunit.client.0.trial144.stderr:441052: waiting osd.3 seq 120259084291 2026-02-19T13:14:30.220 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:14:30.220 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084292 -lt 120259084291 2026-02-19T13:14:30.220 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:14:30.220 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:30.221 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:30.221 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:14:30.221 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:14:30.221 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:14:30.221 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:14:30.221 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:14:30.221 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:14:30.221 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:14:30.221 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:14:30.221 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:14:30.221 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:14:30.221 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:30.222 INFO:tasks.workunit.client.0.trial144.stderr:441052: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:30.222 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:14:30.222 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:14:30.222 INFO:tasks.workunit.client.0.trial144.stderr:441052: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:14:30.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T13:14:30.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T13:14:30.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 441050 2026-02-19T13:14:30.264 INFO:tasks.workunit.client.0.trial144.stderr:h:2267: flush_pg_stats: echo 3-120259084292 2026-02-19T13:14:30.265 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:30.265 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:14:30.265 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-120259084292 2026-02-19T13:14:30.265 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:30.265 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=120259084292 2026-02-19T13:14:30.265 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 120259084292' 2026-02-19T13:14:30.265 INFO:tasks.workunit.client.0.trial144.stderr:441055: waiting osd.3 seq 120259084292 2026-02-19T13:14:30.265 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:14:30.265 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084292 -lt 120259084292 2026-02-19T13:14:30.265 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:14:30.266 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:30.266 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:30.266 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:14:30.266 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:14:30.266 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:14:30.266 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:14:30.266 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:14:30.266 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:14:30.266 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:14:30.266 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:14:30.266 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:14:30.266 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:14:30.266 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:30.267 INFO:tasks.workunit.client.0.trial144.stderr:441055: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:30.267 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:14:30.267 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:14:30.267 INFO:tasks.workunit.client.0.trial144.stderr:441055: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:14:30.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T13:14:30.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T13:14:30.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 441053 2026-02-19T13:14:30.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T13:14:30.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T13:14:30.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T13:14:30.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:940: unfound_erasure_coded: return_code=0 2026-02-19T13:14:30.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:941: unfound_erasure_coded: '[' 0 -ne 0 ']' 2026-02-19T13:14:30.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:946: unfound_erasure_coded: get_pg ecpool SOMETHING 2026-02-19T13:14:30.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T13:14:30.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T13:14:30.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T13:14:30.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:14:30.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:946: unfound_erasure_coded: local pg=1.0 2026-02-19T13:14:30.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:947: unfound_erasure_coded: repair 1.0 2026-02-19T13:14:30.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=1.0 2026-02-19T13:14:30.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 1.0 2026-02-19T13:14:30.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:14:30.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:14:30.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:14:30.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:14:30.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T13:14:19.167520+0000 2026-02-19T13:14:30.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 1.0 2026-02-19T13:14:30.730 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.0s0 on osd.1 to repair 2026-02-19T13:14:30.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 1.0 2026-02-19T13:14:19.167520+0000 2026-02-19T13:14:30.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T13:14:30.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:14:19.167520+0000 2026-02-19T13:14:30.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:14:30.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:14:30.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:14:30.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:14:30.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:14:30.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:14:30.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:14:30.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:14:30.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:14:19.167520+0000 '>' 2026-02-19T13:14:19.167520+0000 2026-02-19T13:14:30.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:14:31.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:14:31.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:14:31.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:14:31.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:14:31.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:14:31.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:14:31.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:14:32.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:14:19.167520+0000 '>' 2026-02-19T13:14:19.167520+0000 2026-02-19T13:14:32.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:14:33.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:14:33.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:14:33.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:14:33.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:14:33.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:14:33.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:14:33.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:14:33.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:14:19.167520+0000 '>' 2026-02-19T13:14:19.167520+0000 2026-02-19T13:14:33.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:14:34.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:14:34.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:14:34.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:14:34.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:14:34.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:14:34.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:14:34.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:14:34.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:14:19.167520+0000 '>' 2026-02-19T13:14:19.167520+0000 2026-02-19T13:14:34.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:14:35.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:14:35.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:14:35.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:14:35.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:14:35.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:14:35.385 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:14:35.385 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:14:35.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:14:19.167520+0000 '>' 2026-02-19T13:14:19.167520+0000 2026-02-19T13:14:35.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:14:36.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:14:36.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:14:36.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:14:36.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:14:36.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:14:36.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:14:36.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:14:36.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:14:31.549025+0000 '>' 2026-02-19T13:14:19.167520+0000 2026-02-19T13:14:36.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:14:36.700 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:952: unfound_erasure_coded: seq 1 60 2026-02-19T13:14:36.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:952: unfound_erasure_coded: for f in `seq 1 60` 2026-02-19T13:14:36.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:953: unfound_erasure_coded: ceph -s 2026-02-19T13:14:36.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:953: unfound_erasure_coded: grep '1/1 objects unfound' 2026-02-19T13:14:36.896 INFO:tasks.workunit.client.0.trial144.stdout: 1/1 objects unfound (100.000%) 2026-02-19T13:14:36.897 INFO:tasks.workunit.client.0.trial144.stdout: 1/1 objects unfound (100.000%) 2026-02-19T13:14:36.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:953: unfound_erasure_coded: break 2026-02-19T13:14:36.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:956: unfound_erasure_coded: ceph -s 2026-02-19T13:14:36.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:956: unfound_erasure_coded: grep '4 up' 2026-02-19T13:14:37.081 INFO:tasks.workunit.client.0.trial144.stdout: osd: 4 osds: 4 up (since 9s), 4 in (since 21s) 2026-02-19T13:14:37.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: unfound_erasure_coded: ceph -s 2026-02-19T13:14:37.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: unfound_erasure_coded: grep '4 in' 2026-02-19T13:14:37.274 INFO:tasks.workunit.client.0.trial144.stdout: osd: 4 osds: 4 up (since 10s), 4 in (since 22s) 2026-02-19T13:14:37.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: unfound_erasure_coded: ceph -s 2026-02-19T13:14:37.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: unfound_erasure_coded: grep '1/1 objects unfound' 2026-02-19T13:14:37.468 INFO:tasks.workunit.client.0.trial144.stdout: 1/1 objects unfound (100.000%) 2026-02-19T13:14:37.468 INFO:tasks.workunit.client.0.trial144.stdout: 1/1 objects unfound (100.000%) 2026-02-19T13:14:37.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T13:14:37.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:14:37.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:14:37.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:14:37.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:14:37.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:14:37.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:14:37.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:14:37.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:14:37.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:14:37.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:14:37.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:14:37.584 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:14:37.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:14:37.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:14:37.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:14:37.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:14:37.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:14:37.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:14:37.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:14:37.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:14:37.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:14:37.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:14:37.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:14:37.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:37.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:37.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:14:37.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:14:37.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:14:37.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-19T13:14:37.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-19T13:14:37.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-repair 2026-02-19T13:14:37.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-repair 2026-02-19T13:14:37.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:14:37.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:14:37.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:14:37.607 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:14:37.607 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:14:37.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:14:37.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:14:37.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:14:37.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:14:37.610 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:14:37.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:14:37.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:14:37.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:14:37.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:14:37.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:14:37.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:14:37.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:14:37.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:14:37.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:14:37.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:14:37.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:14:37.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:14:37.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:14:37.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:37.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:37.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:14:37.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:14:37.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:14:37.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-repair 2026-02-19T13:14:37.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:14:37.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:37.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:37.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35818 2026-02-19T13:14:37.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:14:37.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:14:37.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:14:37.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-19T13:14:37.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_unfound_erasure_coded_overwrites td/osd-scrub-repair 2026-02-19T13:14:37.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:966: TEST_unfound_erasure_coded_overwrites: '[' true = true ']' 2026-02-19T13:14:37.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:967: TEST_unfound_erasure_coded_overwrites: unfound_erasure_coded td/osd-scrub-repair true 2026-02-19T13:14:37.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:911: unfound_erasure_coded: local dir=td/osd-scrub-repair 2026-02-19T13:14:37.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:912: unfound_erasure_coded: local allow_overwrites=true 2026-02-19T13:14:37.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:913: unfound_erasure_coded: local poolname=ecpool 2026-02-19T13:14:37.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:914: unfound_erasure_coded: local payload=ABCDEF 2026-02-19T13:14:37.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:916: unfound_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-19T13:14:37.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-repair 2026-02-19T13:14:37.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:14:37.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:14:37.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:14:37.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-repair/a 2026-02-19T13:14:37.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-19T13:14:37.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:14:37.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:37.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:37.648 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:37.648 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:37.649 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:37.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:37.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:14:37.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:14:37.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:14:37.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:14:37.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:14:37.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:14:37.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:14:37.681 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:14:37.682 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:14:37.682 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:14:37.683 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:14:37.683 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:37.683 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:37.683 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:14:37.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:14:37.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get fsid 2026-02-19T13:14:37.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:14:37.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:14:37.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:14:37.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:14:37.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:14:37.739 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:14:37.739 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:14:37.740 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:14:37.740 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:14:37.740 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:37.740 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:37.740 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35818/ceph-mon.a.asok 2026-02-19T13:14:37.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:14:37.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35818/ceph-mon.a.asok config get mon_host 2026-02-19T13:14:37.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:917: unfound_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-19T13:14:37.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-repair 2026-02-19T13:14:37.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:14:37.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:14:37.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:14:37.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-repair/x 2026-02-19T13:14:37.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:14:37.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:14:37.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:37.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:37.888 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:37.888 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:37.888 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:37.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:37.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:14:37.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:14:37.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:918: unfound_erasure_coded: seq 0 3 2026-02-19T13:14:37.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:918: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-19T13:14:37.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:919: unfound_erasure_coded: run_osd td/osd-scrub-repair 0 2026-02-19T13:14:37.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:14:37.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:37.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:14:37.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:37.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:14:37.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:14:37.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:37.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:37.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:37.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:14:37.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:14:37.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:37.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:37.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:14:37.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:37.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:37.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:37.915 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:37.915 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:37.915 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:37.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:37.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:37.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:37.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:37.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:37.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:14:37.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:14:37.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:37.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:37.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:37.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:37.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:37.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:37.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:14:37.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:37.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b983fcea-d563-4b2d-9293-42c456d0dcb3 2026-02-19T13:14:37.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b983fcea-d563-4b2d-9293-42c456d0dcb3' 2026-02-19T13:14:37.918 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 b983fcea-d563-4b2d-9293-42c456d0dcb3 2026-02-19T13:14:37.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:37.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC9DJdpKi1JNxAA+dtaGCyN8dKOxVKV2ReyHw== 2026-02-19T13:14:37.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC9DJdpKi1JNxAA+dtaGCyN8dKOxVKV2ReyHw=="}' 2026-02-19T13:14:37.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b983fcea-d563-4b2d-9293-42c456d0dcb3 -i td/osd-scrub-repair/0/new.json 2026-02-19T13:14:38.018 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:14:38.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-19T13:14:38.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC9DJdpKi1JNxAA+dtaGCyN8dKOxVKV2ReyHw== --osd-uuid b983fcea-d563-4b2d-9293-42c456d0dcb3 2026-02-19T13:14:38.046 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:38.041+0000 7f7423f97a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:38.047 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:38.045+0000 7f7423f97a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:38.048 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:38.045+0000 7f7423f97a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:38.049 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:38.045+0000 7f7423f97a40 -1 bdev(0x55a832dbec00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:38.049 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:38.045+0000 7f7423f97a40 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-19T13:14:38.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-19T13:14:38.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:38.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:14:38.801 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:14:38.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:39.005 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:14:39.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:14:39.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:39.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:39.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:39.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:39.029 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:39.025+0000 7f9691494a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:39.030 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:39.025+0000 7f9691494a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:39.031 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:39.029+0000 7f9691494a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:39.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:14:39.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:39.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:14:39.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:39.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:39.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:39.164 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:14:39.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:39.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:39.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:14:39.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:39.417 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:39.413+0000 7f9691494a40 -1 Falling back to public interface 2026-02-19T13:14:39.961 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:39.957+0000 7f9691494a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:14:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:40.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:40.329 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:14:40.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:40.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:40.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:14:40.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:41.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:41.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:41.493 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:14:41.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:14:41.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:41.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:14:41.653 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3732397454,v1:127.0.0.1:6803/3732397454] [v2:127.0.0.1:6804/3732397454,v1:127.0.0.1:6805/3732397454] exists,up b983fcea-d563-4b2d-9293-42c456d0dcb3 2026-02-19T13:14:41.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:41.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:41.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:41.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:918: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-19T13:14:41.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:919: unfound_erasure_coded: run_osd td/osd-scrub-repair 1 2026-02-19T13:14:41.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:14:41.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:41.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:14:41.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:41.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-19T13:14:41.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:14:41.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:41.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:41.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:41.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-19T13:14:41.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-19T13:14:41.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:41.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:41.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:14:41.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:41.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:41.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:41.659 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:41.659 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:41.659 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:41.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:41.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:41.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:41.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:41.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:41.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:14:41.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:14:41.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:41.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:41.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:41.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:41.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:41.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:41.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-19T13:14:41.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:41.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=28cd77e2-6f99-4ec8-889c-b5d92e174f4e 2026-02-19T13:14:41.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 28cd77e2-6f99-4ec8-889c-b5d92e174f4e' 2026-02-19T13:14:41.661 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 28cd77e2-6f99-4ec8-889c-b5d92e174f4e 2026-02-19T13:14:41.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:41.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDBDJdpyFEBKBAARkhW7ts3to16zuetUZ8wQw== 2026-02-19T13:14:41.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDBDJdpyFEBKBAARkhW7ts3to16zuetUZ8wQw=="}' 2026-02-19T13:14:41.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 28cd77e2-6f99-4ec8-889c-b5d92e174f4e -i td/osd-scrub-repair/1/new.json 2026-02-19T13:14:41.833 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:14:41.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-19T13:14:41.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDBDJdpyFEBKBAARkhW7ts3to16zuetUZ8wQw== --osd-uuid 28cd77e2-6f99-4ec8-889c-b5d92e174f4e 2026-02-19T13:14:41.860 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:41.857+0000 7f04b8d98a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:41.862 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:41.857+0000 7f04b8d98a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:41.863 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:41.857+0000 7f04b8d98a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:41.863 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:41.861+0000 7f04b8d98a40 -1 bdev(0x55bac8828c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:41.863 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:41.861+0000 7f04b8d98a40 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-19T13:14:42.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-19T13:14:42.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:42.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:14:42.508 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:14:42.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:42.708 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:14:42.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:14:42.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:42.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:42.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:42.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:42.731 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:42.729+0000 7f377166ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:42.733 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:42.729+0000 7f377166ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:42.734 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:42.729+0000 7f377166ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:42.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:14:42.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:42.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:14:42.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:42.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:42.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:42.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:42.868 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:14:42.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:42.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:14:43.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:43.177 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:43.173+0000 7f377166ea40 -1 Falling back to public interface 2026-02-19T13:14:43.770 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:43.765+0000 7f377166ea40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:14:44.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:44.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:44.027 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:14:44.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:44.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:44.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:14:44.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:45.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:45.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:45.189 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:14:45.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:14:45.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:45.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:14:45.345 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1612935500,v1:127.0.0.1:6811/1612935500] [v2:127.0.0.1:6812/1612935500,v1:127.0.0.1:6813/1612935500] exists,up 28cd77e2-6f99-4ec8-889c-b5d92e174f4e 2026-02-19T13:14:45.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:45.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:45.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:45.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:918: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-19T13:14:45.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:919: unfound_erasure_coded: run_osd td/osd-scrub-repair 2 2026-02-19T13:14:45.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:14:45.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:45.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:14:45.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:45.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:14:45.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:14:45.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:45.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:45.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:45.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T13:14:45.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T13:14:45.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:45.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:45.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:14:45.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:45.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:45.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:45.350 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:45.350 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:45.350 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:45.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:45.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:45.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:45.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:45.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:45.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:14:45.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:14:45.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:45.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:45.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:45.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:45.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:45.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:45.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T13:14:45.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:45.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b633cf12-0e22-4a43-a466-ac3d06d39c4a 2026-02-19T13:14:45.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 b633cf12-0e22-4a43-a466-ac3d06d39c4a' 2026-02-19T13:14:45.353 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 b633cf12-0e22-4a43-a466-ac3d06d39c4a 2026-02-19T13:14:45.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:45.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDFDJdpJtOeFRAA8uMsuikSR/A9N7YnW0UShg== 2026-02-19T13:14:45.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDFDJdpJtOeFRAA8uMsuikSR/A9N7YnW0UShg=="}' 2026-02-19T13:14:45.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b633cf12-0e22-4a43-a466-ac3d06d39c4a -i td/osd-scrub-repair/2/new.json 2026-02-19T13:14:45.515 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:14:45.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-19T13:14:45.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDFDJdpJtOeFRAA8uMsuikSR/A9N7YnW0UShg== --osd-uuid b633cf12-0e22-4a43-a466-ac3d06d39c4a 2026-02-19T13:14:45.542 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:45.537+0000 7f51f2aeba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:45.543 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:45.541+0000 7f51f2aeba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:45.544 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:45.541+0000 7f51f2aeba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:45.545 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:45.541+0000 7f51f2aeba40 -1 bdev(0x55b02b9c0c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:45.545 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:45.541+0000 7f51f2aeba40 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-19T13:14:46.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-19T13:14:46.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:46.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:14:46.176 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T13:14:46.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:46.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:14:46.376 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T13:14:46.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:46.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:46.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:46.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:46.392 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:46.389+0000 7f90ca299a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:46.394 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:46.389+0000 7f90ca299a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:46.394 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:46.389+0000 7f90ca299a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:46.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:14:46.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:46.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:14:46.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:46.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:46.535 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:14:46.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:46.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:46.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:46.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:14:46.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:46.825 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:46.821+0000 7f90ca299a40 -1 Falling back to public interface 2026-02-19T13:14:47.429 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:47.425+0000 7f90ca299a40 -1 osd.2 0 log_to_monitors true 2026-02-19T13:14:47.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:47.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:47.698 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:14:47.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:47.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:47.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:14:47.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:48.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:48.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:48.858 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:14:48.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:14:48.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:48.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:14:49.019 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1782542697,v1:127.0.0.1:6819/1782542697] [v2:127.0.0.1:6820/1782542697,v1:127.0.0.1:6821/1782542697] exists,up b633cf12-0e22-4a43-a466-ac3d06d39c4a 2026-02-19T13:14:49.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:49.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:49.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:49.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:918: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-19T13:14:49.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:919: unfound_erasure_coded: run_osd td/osd-scrub-repair 3 2026-02-19T13:14:49.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-repair 2026-02-19T13:14:49.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:49.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:14:49.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:49.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T13:14:49.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:14:49.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:49.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:49.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:49.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T13:14:49.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T13:14:49.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:49.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:49.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:14:49.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:49.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:49.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:49.028 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:49.028 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:49.028 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:49.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:49.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:49.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:49.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:49.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:49.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:14:49.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:14:49.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:49.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:49.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:49.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:49.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:49.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:49.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T13:14:49.029 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:49.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=25584d7b-e08c-4f6d-839f-50589a9e26f7 2026-02-19T13:14:49.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 25584d7b-e08c-4f6d-839f-50589a9e26f7' 2026-02-19T13:14:49.030 INFO:tasks.workunit.client.0.trial144.stdout:add osd3 25584d7b-e08c-4f6d-839f-50589a9e26f7 2026-02-19T13:14:49.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:49.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDJDJdpYBssAhAAkBZQSLQcpgROnHHarTBNTQ== 2026-02-19T13:14:49.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDJDJdpYBssAhAAkBZQSLQcpgROnHHarTBNTQ=="}' 2026-02-19T13:14:49.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 25584d7b-e08c-4f6d-839f-50589a9e26f7 -i td/osd-scrub-repair/3/new.json 2026-02-19T13:14:49.197 INFO:tasks.workunit.client.0.trial144.stdout:3 2026-02-19T13:14:49.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-19T13:14:49.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDJDJdpYBssAhAAkBZQSLQcpgROnHHarTBNTQ== --osd-uuid 25584d7b-e08c-4f6d-839f-50589a9e26f7 2026-02-19T13:14:49.224 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:49.221+0000 7f42964b0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:49.226 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:49.221+0000 7f42964b0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:49.227 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:49.221+0000 7f42964b0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:49.228 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:49.225+0000 7f42964b0a40 -1 bdev(0x55934f720c00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:49.228 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:49.225+0000 7f42964b0a40 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-19T13:14:49.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-19T13:14:49.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:49.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:14:49.876 INFO:tasks.workunit.client.0.trial144.stdout:adding osd3 key to auth repository 2026-02-19T13:14:49.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:50.070 INFO:tasks.workunit.client.0.trial144.stdout:start osd.3 2026-02-19T13:14:50.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:14:50.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:50.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:50.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:50.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:50.095 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:50.089+0000 7ff2f1bdaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:50.096 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:50.093+0000 7ff2f1bdaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:50.097 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:50.093+0000 7ff2f1bdaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:50.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:14:50.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:50.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:14:50.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:50.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:50.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:50.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:50.233 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:14:50.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:50.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:14:50.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:50.509 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:50.505+0000 7ff2f1bdaa40 -1 Falling back to public interface 2026-02-19T13:14:51.133 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:14:51.129+0000 7ff2f1bdaa40 -1 osd.3 0 log_to_monitors true 2026-02-19T13:14:51.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:51.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:51.391 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:14:51.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:51.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:51.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:14:51.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:52.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:52.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:52.548 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:14:52.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:14:52.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:52.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:14:52.702 INFO:tasks.workunit.client.0.trial144.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2250797528,v1:127.0.0.1:6827/2250797528] [v2:127.0.0.1:6828/2250797528,v1:127.0.0.1:6829/2250797528] exists,up 25584d7b-e08c-4f6d-839f-50589a9e26f7 2026-02-19T13:14:52.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:52.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:52.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:52.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: unfound_erasure_coded: create_ec_pool ecpool true k=2 m=2 2026-02-19T13:14:52.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: create_ec_pool: local pool_name=ecpool 2026-02-19T13:14:52.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: create_ec_pool: shift 2026-02-19T13:14:52.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: create_ec_pool: local allow_overwrites=true 2026-02-19T13:14:52.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2525: create_ec_pool: shift 2026-02-19T13:14:52.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=2 2026-02-19T13:14:52.907 INFO:tasks.workunit.client.0.trial144.stderr:load: isa 2026-02-19T13:14:52.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-19T13:14:52.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-19T13:14:53.212 INFO:tasks.workunit.client.0.trial144.stderr:pool 'ecpool' created 2026-02-19T13:14:53.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:14:54.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: create_ec_pool: '[' true = true ']' 2026-02-19T13:14:54.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2532: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-19T13:14:54.425 INFO:tasks.workunit.client.0.trial144.stderr:set pool 1 allow_ec_overwrites to true 2026-02-19T13:14:54.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2535: create_ec_pool: wait_for_clean 2026-02-19T13:14:54.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:14:54.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:14:54.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:14:54.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:14:54.436 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:14:54.436 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:14:54.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:14:54.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:14:54.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:14:54.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:14:54.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:14:54.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:14:54.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:14:54.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:14:54.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:14:54.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:14:54.721 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:14:54.721 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T13:14:54.721 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T13:14:54.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:14:54.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:54.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:14:54.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-19T13:14:54.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-19T13:14:54.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-19T13:14:54.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:54.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:14:54.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-19T13:14:54.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-19T13:14:54.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-38654705667' 2026-02-19T13:14:54.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:54.873 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:14:54.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542147 2026-02-19T13:14:54.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542147 2026-02-19T13:14:54.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-38654705667 2-60129542147' 2026-02-19T13:14:54.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:54.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:14:55.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-19T13:14:55.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-19T13:14:55.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-38654705667 2-60129542147 3-77309411331' 2026-02-19T13:14:55.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:55.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-19T13:14:55.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:55.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:14:55.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-19T13:14:55.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:55.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-19T13:14:55.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-19T13:14:55.023 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836484 2026-02-19T13:14:55.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:55.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-19T13:14:55.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:14:56.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:14:56.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:56.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-19T13:14:56.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:14:57.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:14:57.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:57.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836484 2026-02-19T13:14:57.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:57.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-19T13:14:57.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:57.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:14:57.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705667 2026-02-19T13:14:57.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:57.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705667 2026-02-19T13:14:57.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-19T13:14:57.523 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705667 2026-02-19T13:14:57.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:14:57.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705668 -lt 38654705667 2026-02-19T13:14:57.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:57.697 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542147 2026-02-19T13:14:57.697 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:57.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:14:57.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542147 2026-02-19T13:14:57.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:57.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542147 2026-02-19T13:14:57.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-19T13:14:57.701 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542147 2026-02-19T13:14:57.701 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:14:57.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-19T13:14:57.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:57.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411331 2026-02-19T13:14:57.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:57.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:14:57.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-77309411331 2026-02-19T13:14:57.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:57.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411331 2026-02-19T13:14:57.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 77309411331' 2026-02-19T13:14:57.872 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.3 seq 77309411331 2026-02-19T13:14:57.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:14:58.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411331 -lt 77309411331 2026-02-19T13:14:58.029 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:14:58.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:58.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:58.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:14:58.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:14:58.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:14:58.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:14:58.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:14:58.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:14:58.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:14:58.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:14:58.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:14:58.383 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:14:58.383 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:58.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:58.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:14:58.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:14:58.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:14:58.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2536: create_ec_pool: return 0 2026-02-19T13:14:58.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:924: unfound_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-19T13:14:58.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-19T13:14:58.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-19T13:14:58.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-19T13:14:58.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-19T13:14:58.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-19T13:14:58.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-19T13:14:58.782 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:14:58.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-19T13:14:58.986 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:14:58.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-19T13:14:58.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-19T13:14:58.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-19T13:14:59.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:926: unfound_erasure_coded: get_primary ecpool SOMETHING 2026-02-19T13:14:59.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=ecpool 2026-02-19T13:14:59.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T13:14:59.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:14:59.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-19T13:14:59.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:926: unfound_erasure_coded: local primary=1 2026-02-19T13:14:59.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:927: unfound_erasure_coded: get_osds ecpool SOMETHING 2026-02-19T13:14:59.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:927: unfound_erasure_coded: sed -e s/1// 2026-02-19T13:14:59.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T13:14:59.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:14:59.187 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-19T13:14:59.187 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:14:59.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-19T13:14:59.348 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:14:59.348 INFO:tasks.workunit.client.0.trial144.stderr:2 2026-02-19T13:14:59.348 INFO:tasks.workunit.client.0.trial144.stderr:3' 2026-02-19T13:14:59.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2 3 2026-02-19T13:14:59.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:927: unfound_erasure_coded: osds=('0' '2' '3') 2026-02-19T13:14:59.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:927: unfound_erasure_coded: local -a osds 2026-02-19T13:14:59.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:928: unfound_erasure_coded: local not_primary_first=0 2026-02-19T13:14:59.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:929: unfound_erasure_coded: local not_primary_second=2 2026-02-19T13:14:59.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:930: unfound_erasure_coded: local not_primary_third=3 2026-02-19T13:14:59.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:935: unfound_erasure_coded: pids= 2026-02-19T13:14:59.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:936: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T13:14:59.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T13:14:59.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T13:14:59.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 451699"' 2026-02-19T13:14:59.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 451699' 2026-02-19T13:14:59.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:937: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T13:14:59.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T13:14:59.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T13:14:59.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 451700"' 2026-02-19T13:14:59.350 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 451700' 2026-02-19T13:14:59.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:938: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T13:14:59.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2183: run_in_background: local pid_variable=pids 2026-02-19T13:14:59.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2184: run_in_background: shift 2026-02-19T13:14:59.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: eval 'pids+=" 451702"' 2026-02-19T13:14:59.351 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2188: run_in_background: pids+=' 451702' 2026-02-19T13:14:59.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: unfound_erasure_coded: wait_background pids 2026-02-19T13:14:59.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2219: wait_background: pids=' 451699 451700 451702' 2026-02-19T13:14:59.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2221: wait_background: return_code=0 2026-02-19T13:14:59.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T13:14:59.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 451699 2026-02-19T13:14:59.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T13:14:59.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/451703: /' 2026-02-19T13:14:59.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T13:14:59.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/451705: /' 2026-02-19T13:14:59.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T13:14:59.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: sed 's/^/451707: /' 2026-02-19T13:14:59.968 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:14:59.968 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:14:59.968 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-19T13:14:59.968 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:14:59.969 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T13:14:59.969 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:14:59.969 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:14:59.969 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-19T13:14:59.969 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:14:59.969 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-19T13:14:59.969 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:14:59.969 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:14:59.969 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:14:59.969 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:14:59.969 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:14:59.969 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:14:59.970 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-19T13:14:59.970 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:14:59.970 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:14:59.970 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-19T13:14:59.970 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:14:59.970 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:14:59.970 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-19T13:14:59.970 INFO:tasks.workunit.client.0.trial144.stderr:451703: remove 1#1:eb822e21:::SOMETHING:head# 2026-02-19T13:14:59.970 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-19T13:14:59.970 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:14:59.970 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:14:59.970 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:14:59.970 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:14:59.971 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-19T13:14:59.971 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:14:59.971 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:59.972 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:59.972 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:59.973 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-19T13:14:59.973 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-19T13:14:59.973 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:59.973 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:14:59.973 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:14:59.973 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:14:59.973 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:59.973 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:59.973 INFO:tasks.workunit.client.0.trial144.stderr:451703: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:59.973 INFO:tasks.workunit.client.0.trial144.stderr:451703: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:59.973 INFO:tasks.workunit.client.0.trial144.stderr:451703: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:14:59.974 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:59.974 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:14:59.974 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:59.974 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:14:59.974 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:14:59.974 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:59.974 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:59.974 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:59.974 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:59.974 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:14:59.974 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:14:59.974 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-19T13:14:59.975 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:14:59.975 INFO:tasks.workunit.client.0.trial144.stderr:451703: start osd.0 2026-02-19T13:14:59.975 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:00.012 INFO:tasks.workunit.client.0.trial144.stderr:4451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:15:00.012 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:15:00.012 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T13:15:00.012 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:15:00.012 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T13:15:00.012 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:15:00.012 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:15:00.012 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T13:15:00.013 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:15:00.013 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-19T13:15:00.013 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:15:00.013 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:15:00.013 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:15:00.013 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:15:00.013 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:15:00.013 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:15:00.013 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-19T13:15:00.013 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:15:00.013 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:15:00.013 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T13:15:00.014 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:15:00.014 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-19T13:15:00.014 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-19T13:15:00.014 INFO:tasks.workunit.client.0.trial144.stderr:451707: remove 3#1:eb822e21:::SOMETHING:head# 2026-02-19T13:15:00.014 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-19T13:15:00.014 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:15:00.014 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:15:00.014 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T13:15:00.014 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:15:00.014 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-19T13:15:00.014 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:15:00.014 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:00.016 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:00.016 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:00.016 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-19T13:15:00.016 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-19T13:15:00.016 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:00.016 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:15:00.016 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:15:00.017 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:15:00.017 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:00.017 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:00.017 INFO:tasks.workunit.client.0.trial144.stderr:451707: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:00.017 INFO:tasks.workunit.client.0.trial144.stderr:451707: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:00.017 INFO:tasks.workunit.client.0.trial144.stderr:451707: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:15:00.017 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:15:00.017 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:15:00.017 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:00.017 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:15:00.017 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:15:00.017 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:00.017 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:00.018 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:00.018 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:00.018 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:15:00.018 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:15:00.018 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-19T13:15:00.018 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T13:15:00.018 INFO:tasks.workunit.client.0.trial144.stderr:451707: start osd.3 2026-02-19T13:15:00.018 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:00.048 INFO:tasks.workunit.client.0.trial144.stderr:4451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-19T13:15:00.048 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:15:00.048 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T13:15:00.048 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:15:00.048 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T13:15:00.048 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-19T13:15:00.049 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:15:00.049 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T13:15:00.049 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:15:00.049 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-19T13:15:00.049 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:15:00.049 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:15:00.049 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:15:00.049 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:15:00.049 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:15:00.049 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:15:00.049 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING remove 2026-02-19T13:15:00.049 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-19T13:15:00.050 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:15:00.050 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T13:15:00.050 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:15:00.050 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:15:00.050 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING remove 2026-02-19T13:15:00.050 INFO:tasks.workunit.client.0.trial144.stderr:451705: remove 2#1:eb822e21:::SOMETHING:head# 2026-02-19T13:15:00.050 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-19T13:15:00.050 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-repair 2026-02-19T13:15:00.050 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:15:00.050 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:15:00.050 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:15:00.050 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-19T13:15:00.050 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-19T13:15:00.051 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:00.052 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:00.052 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:00.052 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-19T13:15:00.052 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-19T13:15:00.052 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:00.052 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:15:00.052 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-19T13:15:00.053 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:15:00.053 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:00.053 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:00.053 INFO:tasks.workunit.client.0.trial144.stderr:451705: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:00.053 INFO:tasks.workunit.client.0.trial144.stderr:451705: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:00.053 INFO:tasks.workunit.client.0.trial144.stderr:451705: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:15:00.053 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:15:00.053 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' 2026-02-19T13:15:00.053 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:00.053 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-19T13:15:00.053 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-19T13:15:00.053 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:00.054 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:00.054 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:00.054 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:00.054 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:15:00.054 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:15:00.054 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-19T13:15:00.054 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:15:00.054 INFO:tasks.workunit.client.0.trial144.stderr:451705: start osd.2 2026-02-19T13:15:00.054 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1c99343c-e9bd-4db1-819d-e7e37994f3b3 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.35818/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:01.482 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-19T13:15:01.482 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:15:01.482 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:15:01.483 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:15:01.483 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:15:01.483 INFO:tasks.workunit.client.0.trial144.stderr:451703: 2026-02-19T13:14:59.989+0000 7f3dc36f9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:01.483 INFO:tasks.workunit.client.0.trial144.stderr:451703: 2026-02-19T13:14:59.989+0000 7f3dc36f9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:01.483 INFO:tasks.workunit.client.0.trial144.stderr:451703: 2026-02-19T13:14:59.989+0000 7f3dc36f9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:01.483 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:15:01.483 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:01.483 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:15:01.483 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:01.483 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:01.483 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:01.483 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:01.484 INFO:tasks.workunit.client.0.trial144.stderr:451703: 0 2026-02-19T13:15:01.484 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:01.484 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:15:01.484 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:01.484 INFO:tasks.workunit.client.0.trial144.stderr:451703: 2026-02-19T13:15:00.533+0000 7f3dc36f9a40 -1 Falling back to public interface 2026-02-19T13:15:01.484 INFO:tasks.workunit.client.0.trial144.stderr:451703: 2026-02-19T13:15:01.237+0000 7f3dc36f9a40 -1 osd.0 27 log_to_monitors true 2026-02-19T13:15:01.484 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:01.484 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:01.484 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:01.484 INFO:tasks.workunit.client.0.trial144.stderr:451703: 1 2026-02-19T13:15:01.484 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:01.484 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:15:01.485 INFO:tasks.workunit.client.0.trial144.stderr:451703: osd.0 up in weight 1 up_from 31 up_thru 0 down_at 28 last_clean_interval [5,27) [v2:127.0.0.1:6802/1834819226,v1:127.0.0.1:6803/1834819226] [v2:127.0.0.1:6804/1834819226,v1:127.0.0.1:6805/1834819226] exists,up b983fcea-d563-4b2d-9293-42c456d0dcb3 2026-02-19T13:15:01.485 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:01.485 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:01.485 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:01.485 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:15:01.485 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:15:01.485 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:15:01.485 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:15:01.485 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:15:01.485 INFO:tasks.workunit.client.0.trial144.stderr:451703: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:15:01.520 INFO:tasks.workunit.client.0.trial144.stderr:451703: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get51707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-19T13:15:01.520 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T13:15:01.521 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:15:01.521 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:15:01.521 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:15:01.521 INFO:tasks.workunit.client.0.trial144.stderr:451707: 2026-02-19T13:15:00.033+0000 7fc7e1372a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:01.521 INFO:tasks.workunit.client.0.trial144.stderr:451707: 2026-02-19T13:15:00.033+0000 7fc7e1372a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:01.521 INFO:tasks.workunit.client.0.trial144.stderr:451707: 2026-02-19T13:15:00.033+0000 7fc7e1372a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:01.521 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T13:15:01.521 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:01.521 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:15:01.521 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:01.521 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:01.521 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:01.522 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:01.522 INFO:tasks.workunit.client.0.trial144.stderr:451707: 0 2026-02-19T13:15:01.522 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:01.522 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:15:01.522 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:01.522 INFO:tasks.workunit.client.0.trial144.stderr:451707: 2026-02-19T13:15:00.565+0000 7fc7e1372a40 -1 Falling back to public interface 2026-02-19T13:15:01.522 INFO:tasks.workunit.client.0.trial144.stderr:451707: 2026-02-19T13:15:01.281+0000 7fc7e1372a40 -1 osd.3 27 log_to_monitors true 2026-02-19T13:15:01.522 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:01.522 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:01.522 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:01.522 INFO:tasks.workunit.client.0.trial144.stderr:451707: 1 2026-02-19T13:15:01.522 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:01.523 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:15:01.523 INFO:tasks.workunit.client.0.trial144.stderr:451707: osd.3 up in weight 1 up_from 31 up_thru 0 down_at 28 last_clean_interval [18,27) [v2:127.0.0.1:6826/915517247,v1:127.0.0.1:6827/915517247] [v2:127.0.0.1:6828/915517247,v1:127.0.0.1:6829/915517247] exists,up 25584d7b-e08c-4f6d-839f-50589a9e26f7 2026-02-19T13:15:01.523 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:01.523 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:01.523 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:01.523 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:15:01.523 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:15:01.523 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:15:01.523 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:15:01.523 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:15:01.523 INFO:tasks.workunit.client.0.trial144.stderr:451707: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:15:01.544 INFO:tasks.workunit.client.0.trial144.stderr:451707: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_ti51705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-19T13:15:01.544 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:15:01.544 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:15:01.544 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:15:01.545 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:15:01.545 INFO:tasks.workunit.client.0.trial144.stderr:451705: 2026-02-19T13:15:00.065+0000 7f5a9c672a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:01.545 INFO:tasks.workunit.client.0.trial144.stderr:451705: 2026-02-19T13:15:00.065+0000 7f5a9c672a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:01.545 INFO:tasks.workunit.client.0.trial144.stderr:451705: 2026-02-19T13:15:00.065+0000 7f5a9c672a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:01.545 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:15:01.545 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:01.545 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:15:01.545 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:01.545 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:01.545 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:01.545 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:01.545 INFO:tasks.workunit.client.0.trial144.stderr:451705: 0 2026-02-19T13:15:01.545 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial144.stderr:451705: 2026-02-19T13:15:00.553+0000 7f5a9c672a40 -1 Falling back to public interface 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial144.stderr:451705: 2026-02-19T13:15:01.293+0000 7f5a9c672a40 -1 osd.2 27 log_to_monitors true 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial144.stderr:451705: 1 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial144.stderr:451705: osd.2 up in weight 1 up_from 32 up_thru 0 down_at 28 last_clean_interval [14,27) [v2:127.0.0.1:6818/315959689,v1:127.0.0.1:6819/315959689] [v2:127.0.0.1:6820/315959689,v1:127.0.0.1:6821/315959689] exists,up b633cf12-0e22-4a43-a466-ac3d06d39c4a 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:01.547 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:01.547 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:15:01.547 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:15:01.547 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:15:01.547 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:15:01.547 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:15:01.547 INFO:tasks.workunit.client.0.trial144.stderr:451705: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:15:02.062 INFO:tasks.workunit.client.0.trial144.stderr:451705: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_ti_timeout_delays: echo true 2026-02-19T13:15:02.062 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:15:02.062 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:15:02.062 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:15:02.062 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:15:02.062 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:15:02.062 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:15:02.062 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:15:02.063 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:15:02.063 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:15:02.063 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:15:02.063 INFO:tasks.workunit.client.0.trial144.stderr:451703: 1 2026-02-19T13:15:02.063 INFO:tasks.workunit.client.0.trial144.stderr:451703: 2 2026-02-19T13:15:02.063 INFO:tasks.workunit.client.0.trial144.stderr:451703: 3' 2026-02-19T13:15:02.063 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:15:02.063 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:02.063 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:15:02.063 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=133143986178 2026-02-19T13:15:02.063 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 133143986178 2026-02-19T13:15:02.063 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986178' 2026-02-19T13:15:02.064 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:02.064 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:15:02.064 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705670 2026-02-19T13:15:02.064 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705670 2026-02-19T13:15:02.064 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986178 1-38654705670' 2026-02-19T13:15:02.064 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:02.064 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:15:02.064 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=137438953474 2026-02-19T13:15:02.064 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 137438953474 2026-02-19T13:15:02.064 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986178 1-38654705670 2-137438953474' 2026-02-19T13:15:02.064 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:02.064 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:15:02.064 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=133143986178 2026-02-19T13:15:02.065 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 133143986178 2026-02-19T13:15:02.065 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986178 1-38654705670 2-137438953474 3-133143986178' 2026-02-19T13:15:02.065 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:02.065 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-133143986178 2026-02-19T13:15:02.095 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/meout_delays: echo true 2026-02-19T13:15:02.096 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:15:02.096 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:15:02.096 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:15:02.096 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:15:02.096 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:15:02.096 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:15:02.096 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:15:02.096 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:15:02.097 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:15:02.097 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:15:02.097 INFO:tasks.workunit.client.0.trial144.stderr:451707: 1 2026-02-19T13:15:02.097 INFO:tasks.workunit.client.0.trial144.stderr:451707: 2 2026-02-19T13:15:02.097 INFO:tasks.workunit.client.0.trial144.stderr:451707: 3' 2026-02-19T13:15:02.097 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:15:02.097 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:02.097 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:15:02.097 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=133143986179 2026-02-19T13:15:02.097 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 133143986179 2026-02-19T13:15:02.097 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986179' 2026-02-19T13:15:02.097 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:02.098 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:15:02.098 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705671 2026-02-19T13:15:02.098 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705671 2026-02-19T13:15:02.098 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986179 1-38654705671' 2026-02-19T13:15:02.098 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:02.098 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:15:02.098 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=137438953475 2026-02-19T13:15:02.098 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 137438953475 2026-02-19T13:15:02.098 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986179 1-38654705671 2-137438953475' 2026-02-19T13:15:02.098 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:02.098 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:15:02.098 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=133143986179 2026-02-19T13:15:02.099 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 133143986179 2026-02-19T13:15:02.099 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986179 1-38654705671 2-137438953475 3-133143986179' 2026-02-19T13:15:02.099 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:02.099 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-133143986179 2026-02-19T13:15:02.136 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubumeout_delays: echo true 2026-02-19T13:15:02.136 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:15:02.136 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:15:02.136 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:15:02.136 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:15:02.136 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:15:02.136 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:15:02.136 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:15:02.136 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:15:02.136 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:15:02.137 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:15:02.137 INFO:tasks.workunit.client.0.trial144.stderr:451705: 1 2026-02-19T13:15:02.137 INFO:tasks.workunit.client.0.trial144.stderr:451705: 2 2026-02-19T13:15:02.137 INFO:tasks.workunit.client.0.trial144.stderr:451705: 3' 2026-02-19T13:15:02.137 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:15:02.137 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:02.137 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:15:02.137 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=133143986180 2026-02-19T13:15:02.137 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 133143986180 2026-02-19T13:15:02.137 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986180' 2026-02-19T13:15:02.137 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:02.137 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:15:02.137 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705672 2026-02-19T13:15:02.138 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705672 2026-02-19T13:15:02.138 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986180 1-38654705672' 2026-02-19T13:15:02.138 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:02.138 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:15:02.138 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=137438953476 2026-02-19T13:15:02.138 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 137438953476 2026-02-19T13:15:02.138 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986180 1-38654705672 2-137438953476' 2026-02-19T13:15:02.138 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:02.138 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:15:02.138 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=133143986180 2026-02-19T13:15:02.138 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 133143986180 2026-02-19T13:15:02.138 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986180 1-38654705672 2-137438953476 3-133143986180' 2026-02-19T13:15:02.138 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:02.139 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-133143986180 2026-02-19T13:15:03.739 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:03.739 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:15:03.739 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-133143986178 2026-02-19T13:15:03.739 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:03.739 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=133143986178 2026-02-19T13:15:03.739 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 133143986178' 2026-02-19T13:15:03.740 INFO:tasks.workunit.client.0.trial144.stderr:451703: waiting osd.0 seq 133143986178 2026-02-19T13:15:03.740 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:03.740 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 133143986178 2026-02-19T13:15:03.740 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:15:03.740 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:15:03.740 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:03.740 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 133143986180 -lt 133143986178 2026-02-19T13:15:03.740 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:03.740 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705670 2026-02-19T13:15:03.740 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:03.740 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:15:03.740 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705670 2026-02-19T13:15:03.740 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:03.741 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705670 2026-02-19T13:15:03.741 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-19T13:15:03.741 INFO:tasks.workunit.client.0.trial144.stderr:451703: waiting osd.1 seq 38654705670 2026-02-19T13:15:03.741 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:15:03.741 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705672 -lt 38654705670 2026-02-19T13:15:03.741 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:03.741 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-137438953474 2026-02-19T13:15:03.741 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:03.741 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:15:03.741 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-137438953474 2026-02-19T13:15:03.741 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:03.741 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=137438953474 2026-02-19T13:15:03.741 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 137438953474' 2026-02-19T13:15:03.742 INFO:tasks.workunit.client.0.trial144.stderr:451703: waiting osd.2 seq 137438953474 2026-02-19T13:15:03.742 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:15:03.742 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 137438953476 -lt 137438953474 2026-02-19T13:15:03.742 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:03.772 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.shntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:03.772 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:15:03.772 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-133143986179 2026-02-19T13:15:03.772 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:03.772 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=133143986179 2026-02-19T13:15:03.772 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 133143986179' 2026-02-19T13:15:03.772 INFO:tasks.workunit.client.0.trial144.stderr:451707: waiting osd.0 seq 133143986179 2026-02-19T13:15:03.772 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:03.772 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 133143986179 2026-02-19T13:15:03.772 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:15:03.773 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:15:03.773 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:03.773 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 133143986180 -lt 133143986179 2026-02-19T13:15:03.773 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:03.773 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705671 2026-02-19T13:15:03.773 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:03.773 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:15:03.773 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705671 2026-02-19T13:15:03.773 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:03.773 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705671 2026-02-19T13:15:03.773 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-19T13:15:03.773 INFO:tasks.workunit.client.0.trial144.stderr:451707: waiting osd.1 seq 38654705671 2026-02-19T13:15:03.773 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:15:03.774 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705672 -lt 38654705671 2026-02-19T13:15:03.774 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:03.774 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-137438953475 2026-02-19T13:15:03.774 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:03.774 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:15:03.774 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-137438953475 2026-02-19T13:15:03.774 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:03.774 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=137438953475 2026-02-19T13:15:03.774 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 137438953475' 2026-02-19T13:15:03.774 INFO:tasks.workunit.client.0.trial144.stderr:451707: waiting osd.2 seq 137438953475 2026-02-19T13:15:03.774 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:15:03.774 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 137438953476 -lt 137438953475 2026-02-19T13:15:03.774 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:03.830 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:22ntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:03.830 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:15:03.831 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-133143986180 2026-02-19T13:15:03.831 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:03.831 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=133143986180 2026-02-19T13:15:03.831 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 133143986180' 2026-02-19T13:15:03.831 INFO:tasks.workunit.client.0.trial144.stderr:451705: waiting osd.0 seq 133143986180 2026-02-19T13:15:03.831 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:03.831 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 133143986180 2026-02-19T13:15:03.831 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:15:03.831 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:15:03.831 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:03.831 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 133143986180 -lt 133143986180 2026-02-19T13:15:03.831 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:03.832 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705672 2026-02-19T13:15:03.832 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:03.832 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:15:03.832 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705672 2026-02-19T13:15:03.832 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:03.832 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705672 2026-02-19T13:15:03.832 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705672' 2026-02-19T13:15:03.832 INFO:tasks.workunit.client.0.trial144.stderr:451705: waiting osd.1 seq 38654705672 2026-02-19T13:15:03.832 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:15:03.832 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705672 -lt 38654705672 2026-02-19T13:15:03.832 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:03.832 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-137438953476 2026-02-19T13:15:03.832 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:03.833 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:15:03.833 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-137438953476 2026-02-19T13:15:03.833 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:03.833 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=137438953476 2026-02-19T13:15:03.833 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 137438953476' 2026-02-19T13:15:03.833 INFO:tasks.workunit.client.0.trial144.stderr:451705: waiting osd.2 seq 137438953476 2026-02-19T13:15:03.833 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:15:03.833 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 137438953476 -lt 137438953476 2026-02-19T13:15:03.833 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:04.476 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:22:2267: flush_pg_stats: echo 3-133143986178 2026-02-19T13:15:04.476 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:04.476 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:15:04.477 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-133143986178 2026-02-19T13:15:04.477 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:04.477 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=133143986178 2026-02-19T13:15:04.477 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 133143986178' 2026-02-19T13:15:04.477 INFO:tasks.workunit.client.0.trial144.stderr:451703: waiting osd.3 seq 133143986178 2026-02-19T13:15:04.477 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:15:04.477 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 133143986180 -lt 133143986178 2026-02-19T13:15:04.477 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:15:04.477 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:04.477 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:04.477 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:15:04.478 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:15:04.478 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:15:04.478 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:15:04.478 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:15:04.478 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:15:04.478 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:15:04.478 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:15:04.478 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:15:04.478 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:15:04.478 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:04.478 INFO:tasks.workunit.client.0.trial144.stderr:451703: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:04.478 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:15:04.479 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:15:04.479 INFO:tasks.workunit.client.0.trial144.stderr:451703: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:15:04.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T13:15:04.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T13:15:04.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 451700 2026-02-19T13:15:04.487 INFO:tasks.workunit.client.0.trial144.stderr:67: flush_pg_stats: echo 3-133143986179 2026-02-19T13:15:04.487 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:04.487 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:15:04.487 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-133143986179 2026-02-19T13:15:04.488 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:04.488 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=133143986179 2026-02-19T13:15:04.488 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 133143986179' 2026-02-19T13:15:04.488 INFO:tasks.workunit.client.0.trial144.stderr:451707: waiting osd.3 seq 133143986179 2026-02-19T13:15:04.488 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:15:04.488 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 133143986180 -lt 133143986179 2026-02-19T13:15:04.488 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:15:04.488 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:04.488 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:04.488 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:15:04.488 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:15:04.488 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:15:04.489 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:15:04.489 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:15:04.489 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:15:04.489 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:15:04.489 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:15:04.489 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:15:04.489 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:15:04.489 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:04.489 INFO:tasks.workunit.client.0.trial144.stderr:451707: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:04.489 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:15:04.489 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:15:04.489 INFO:tasks.workunit.client.0.trial144.stderr:451707: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:15:04.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T13:15:04.555 INFO:tasks.workunit.client.0.trial144.stderr:67: flush_pg_stats: echo 3-133143986180 2026-02-19T13:15:04.555 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:04.555 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:15:04.555 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-133143986180 2026-02-19T13:15:04.555 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:04.555 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=133143986180 2026-02-19T13:15:04.555 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 133143986180' 2026-02-19T13:15:04.555 INFO:tasks.workunit.client.0.trial144.stderr:451705: waiting osd.3 seq 133143986180 2026-02-19T13:15:04.556 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:15:04.556 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 133143986180 -lt 133143986180 2026-02-19T13:15:04.556 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:15:04.556 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:04.556 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:04.556 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:15:04.556 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:15:04.556 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:15:04.556 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:15:04.556 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:15:04.556 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:15:04.556 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:15:04.557 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:15:04.557 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:15:04.557 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:15:04.557 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:04.557 INFO:tasks.workunit.client.0.trial144.stderr:451705: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:04.557 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:15:04.557 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:15:04.557 INFO:tasks.workunit.client.0.trial144.stderr:451705: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:15:04.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2187: run_in_background: return 0 2026-02-19T13:15:04.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2222: wait_background: for pid in $pids 2026-02-19T13:15:04.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2223: wait_background: wait 451702 2026-02-19T13:15:04.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: eval 'pids='\'''\''' 2026-02-19T13:15:04.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: pids= 2026-02-19T13:15:04.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2232: wait_background: return 0 2026-02-19T13:15:04.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:940: unfound_erasure_coded: return_code=0 2026-02-19T13:15:04.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:941: unfound_erasure_coded: '[' 0 -ne 0 ']' 2026-02-19T13:15:04.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:946: unfound_erasure_coded: get_pg ecpool SOMETHING 2026-02-19T13:15:04.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=ecpool 2026-02-19T13:15:04.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T13:15:04.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-19T13:15:04.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:15:04.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:946: unfound_erasure_coded: local pg=1.0 2026-02-19T13:15:04.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:947: unfound_erasure_coded: repair 1.0 2026-02-19T13:15:04.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=1.0 2026-02-19T13:15:04.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 1.0 2026-02-19T13:15:04.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:15:04.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:15:04.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:15:04.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:15:04.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T13:14:53.212248+0000 2026-02-19T13:15:04.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 1.0 2026-02-19T13:15:05.030 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.0s0 on osd.1 to repair 2026-02-19T13:15:05.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 1.0 2026-02-19T13:14:53.212248+0000 2026-02-19T13:15:05.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T13:15:05.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:14:53.212248+0000 2026-02-19T13:15:05.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:15:05.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:15:05.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:15:05.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:15:05.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:15:05.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:15:05.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:15:05.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:15:05.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:14:53.212248+0000 '>' 2026-02-19T13:14:53.212248+0000 2026-02-19T13:15:05.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:15:06.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:15:06.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:15:06.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:15:06.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:15:06.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:15:06.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:15:06.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:15:06.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:14:53.212248+0000 '>' 2026-02-19T13:14:53.212248+0000 2026-02-19T13:15:06.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:15:07.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:15:07.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:15:07.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:15:07.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:15:07.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:15:07.369 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:15:07.369 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:15:07.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:14:53.212248+0000 '>' 2026-02-19T13:14:53.212248+0000 2026-02-19T13:15:07.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:15:08.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:15:08.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:15:08.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:15:08.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:15:08.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:15:08.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:15:08.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:15:08.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:14:53.212248+0000 '>' 2026-02-19T13:14:53.212248+0000 2026-02-19T13:15:08.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:15:09.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:15:09.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:15:09.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:15:09.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:15:09.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:15:09.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:15:09.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:15:09.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:14:53.212248+0000 '>' 2026-02-19T13:14:53.212248+0000 2026-02-19T13:15:09.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:15:10.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:15:10.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:15:10.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:15:10.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:15:10.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:15:10.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:15:10.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:15:11.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:15:05.717942+0000 '>' 2026-02-19T13:14:53.212248+0000 2026-02-19T13:15:11.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:15:11.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:952: unfound_erasure_coded: seq 1 60 2026-02-19T13:15:11.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:952: unfound_erasure_coded: for f in `seq 1 60` 2026-02-19T13:15:11.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:953: unfound_erasure_coded: ceph -s 2026-02-19T13:15:11.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:953: unfound_erasure_coded: grep '1/1 objects unfound' 2026-02-19T13:15:11.210 INFO:tasks.workunit.client.0.trial144.stdout: 1/1 objects unfound (100.000%) 2026-02-19T13:15:11.210 INFO:tasks.workunit.client.0.trial144.stdout: 1/1 objects unfound (100.000%) 2026-02-19T13:15:11.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:953: unfound_erasure_coded: break 2026-02-19T13:15:11.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:956: unfound_erasure_coded: ceph -s 2026-02-19T13:15:11.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:956: unfound_erasure_coded: grep '4 up' 2026-02-19T13:15:11.404 INFO:tasks.workunit.client.0.trial144.stdout: osd: 4 osds: 4 up (since 9s), 4 in (since 22s) 2026-02-19T13:15:11.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: unfound_erasure_coded: ceph -s 2026-02-19T13:15:11.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: unfound_erasure_coded: grep '4 in' 2026-02-19T13:15:11.600 INFO:tasks.workunit.client.0.trial144.stdout: osd: 4 osds: 4 up (since 10s), 4 in (since 22s) 2026-02-19T13:15:11.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: unfound_erasure_coded: ceph -s 2026-02-19T13:15:11.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: unfound_erasure_coded: grep '1/1 objects unfound' 2026-02-19T13:15:11.795 INFO:tasks.workunit.client.0.trial144.stdout: 1/1 objects unfound (100.000%) 2026-02-19T13:15:11.796 INFO:tasks.workunit.client.0.trial144.stdout: 1/1 objects unfound (100.000%) 2026-02-19T13:15:11.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-19T13:15:11.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:15:11.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:15:11.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:15:11.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:15:11.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:15:11.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:15:11.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:15:11.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:15:11.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:15:11.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:15:11.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:15:11.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:15:11.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:15:11.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:15:11.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:15:11.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:11.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:15:11.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:15:11.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:11.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:15:11.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:15:11.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:15:11.932 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:15:11.932 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:11.932 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:15:11.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:15:11.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:15:11.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:15:11.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-19T13:15:11.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/osd-scrub-repair 0 2026-02-19T13:15:11.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-repair 2026-02-19T13:15:11.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-19T13:15:11.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-19T13:15:11.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:15:11.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:15:11.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:15:11.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:15:11.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:15:11.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:15:11.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:15:11.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:15:11.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:15:11.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:15:11.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:15:11.942 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:15:11.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:11.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:15:11.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:15:11.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:11.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:15:11.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-19T13:15:11.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-repair 2026-02-19T13:15:11.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:15:11.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:11.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35818 2026-02-19T13:15:11.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35818 2026-02-19T13:15:11.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:15:11.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:15:11.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-19T13:15:11.952 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-19T13:15:11.952 DEBUG:teuthology.orchestra.run.trial144:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-19T13:15:11.961 INFO:tasks.workunit:Running workunit scrub/osd-scrub-snaps.sh... 2026-02-19T13:15:11.961 DEBUG:teuthology.orchestra.run.trial144:workunit test scrub/osd-scrub-snaps.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=624b9a7bcb2064ce17206482df6a1da007ff7675 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh 2026-02-19T13:15:12.013 INFO:tasks.workunit.client.0.trial144.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-19T13:15:12.017 INFO:tasks.workunit.client.0.trial144.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-19T13:15:12.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-19T13:15:12.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-19T13:15:12.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-19T13:15:12.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-19T13:15:12.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-19T13:15:12.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-19T13:15:12.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-19T13:15:12.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-19T13:15:12.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/osd-scrub-snaps 2026-02-19T13:15:12.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:27: run: local dir=td/osd-scrub-snaps 2026-02-19T13:15:12.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:28: run: shift 2026-02-19T13:15:12.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:30: run: export CEPH_MON=127.0.0.1:7121 2026-02-19T13:15:12.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:30: run: CEPH_MON=127.0.0.1:7121 2026-02-19T13:15:12.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:31: run: export CEPH_ARGS 2026-02-19T13:15:12.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:32: run: uuidgen 2026-02-19T13:15:12.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:32: run: CEPH_ARGS+='--fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none ' 2026-02-19T13:15:12.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:33: run: CEPH_ARGS+='--mon-host=127.0.0.1:7121 ' 2026-02-19T13:15:12.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:35: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-19T13:15:12.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:36: run: set 2026-02-19T13:15:12.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:36: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-19T13:15:12.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:36: run: local 'funcs=TEST_scrub_snaps 2026-02-19T13:15:12.022 INFO:tasks.workunit.client.0.trial144.stderr:TEST_scrub_snaps_primary 2026-02-19T13:15:12.022 INFO:tasks.workunit.client.0.trial144.stderr:TEST_scrub_snaps_replica' 2026-02-19T13:15:12.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:37: run: for func in $funcs 2026-02-19T13:15:12.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:38: run: setup td/osd-scrub-snaps 2026-02-19T13:15:12.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-snaps 2026-02-19T13:15:12.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-snaps 2026-02-19T13:15:12.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-snaps 2026-02-19T13:15:12.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:15:12.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-19T13:15:12.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:15:12.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:15:12.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:15:12.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:15:12.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:15:12.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:15:12.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:15:12.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:15:12.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:15:12.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:15:12.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:15:12.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:15:12.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:12.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:15:12.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:15:12.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:12.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:15:12.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:15:12.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-snaps 2026-02-19T13:15:12.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:15:12.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:12.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:12.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.456002 2026-02-19T13:15:12.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:15:12.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:15:12.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-snaps 2026-02-19T13:15:12.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:15:12.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:12.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:12.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.456002 2026-02-19T13:15:12.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:15:12.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-19T13:15:12.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-19T13:15:12.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:15:12.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-snaps 1' TERM HUP INT 2026-02-19T13:15:12.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:39: run: TEST_scrub_snaps td/osd-scrub-snaps 2026-02-19T13:15:12.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:171: TEST_scrub_snaps: local dir=td/osd-scrub-snaps 2026-02-19T13:15:12.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:172: TEST_scrub_snaps: local poolname=test 2026-02-19T13:15:12.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:173: TEST_scrub_snaps: local OBJS=16 2026-02-19T13:15:12.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:174: TEST_scrub_snaps: local OSDS=1 2026-02-19T13:15:12.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:176: TEST_scrub_snaps: TESTDATA=testdata.456002 2026-02-19T13:15:12.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:178: TEST_scrub_snaps: run_mon td/osd-scrub-snaps a --osd_pool_default_size=1 2026-02-19T13:15:12.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-snaps 2026-02-19T13:15:12.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:15:12.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:15:12.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:15:12.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-snaps/a 2026-02-19T13:15:12.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-snaps/a --run-dir=td/osd-scrub-snaps --osd_pool_default_size=1 2026-02-19T13:15:12.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:15:12.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:12.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:12.062 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:12.062 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:12.062 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:12.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:15:12.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-snaps/a '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-snaps/log --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 2026-02-19T13:15:12.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:15:12.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:15:12.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:15:12.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:15:12.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:15:12.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:15:12.095 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:15:12.095 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:15:12.095 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:15:12.095 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:12.095 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:12.095 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:12.095 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.456002/ceph-mon.a.asok 2026-02-19T13:15:12.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:15:12.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.456002/ceph-mon.a.asok config get fsid 2026-02-19T13:15:12.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:15:12.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:15:12.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:15:12.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:15:12.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:15:12.152 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:15:12.152 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:15:12.153 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:15:12.153 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:12.153 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:12.153 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:12.153 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.456002/ceph-mon.a.asok 2026-02-19T13:15:12.153 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:15:12.153 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.456002/ceph-mon.a.asok config get mon_host 2026-02-19T13:15:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:179: TEST_scrub_snaps: run_mgr td/osd-scrub-snaps x 2026-02-19T13:15:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-snaps 2026-02-19T13:15:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:15:12.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:15:12.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:15:12.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-snaps/x 2026-02-19T13:15:12.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:15:12.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:15:12.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:12.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:12.306 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:12.306 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:12.306 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:12.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:15:12.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:15:12.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-snaps/x '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:15:12.325 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:180: TEST_scrub_snaps: expr 1 - 1 2026-02-19T13:15:12.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:180: TEST_scrub_snaps: seq 0 0 2026-02-19T13:15:12.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:180: TEST_scrub_snaps: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:15:12.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:182: TEST_scrub_snaps: run_osd td/osd-scrub-snaps 0 2026-02-19T13:15:12.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-snaps 2026-02-19T13:15:12.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:12.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:15:12.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:12.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-19T13:15:12.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-19T13:15:12.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:12.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:12.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:12.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-19T13:15:12.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-19T13:15:12.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:12.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:12.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-19T13:15:12.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:12.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:12.331 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:12.331 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:12.331 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:12.331 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:12.331 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:15:12.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:15:12.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:12.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:12.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:12.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-19T13:15:12.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-19T13:15:12.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:12.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:12.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:12.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:12.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:12.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:12.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-19T13:15:12.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:12.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=42f5abfb-2b06-4eeb-907a-7398feba962b 2026-02-19T13:15:12.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 42f5abfb-2b06-4eeb-907a-7398feba962b' 2026-02-19T13:15:12.333 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 42f5abfb-2b06-4eeb-907a-7398feba962b 2026-02-19T13:15:12.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:12.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDgDJdpDJxTFBAAJd33+7w8FOhKpSPsJCCFzw== 2026-02-19T13:15:12.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDgDJdpDJxTFBAAJd33+7w8FOhKpSPsJCCFzw=="}' 2026-02-19T13:15:12.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 42f5abfb-2b06-4eeb-907a-7398feba962b -i td/osd-scrub-snaps/0/new.json 2026-02-19T13:15:12.430 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:15:12.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-snaps/0/new.json 2026-02-19T13:15:12.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDgDJdpDJxTFBAAJd33+7w8FOhKpSPsJCCFzw== --osd-uuid 42f5abfb-2b06-4eeb-907a-7398feba962b 2026-02-19T13:15:12.458 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:12.453+0000 7f668cb74a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:12.460 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:12.457+0000 7f668cb74a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:12.460 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:12.457+0000 7f668cb74a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:12.461 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:12.457+0000 7f668cb74a40 -1 bdev(0x55a45ef22c00 td/osd-scrub-snaps/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:12.461 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:12.457+0000 7f668cb74a40 -1 bluestore(td/osd-scrub-snaps/0) _read_fsid unparsable uuid 2026-02-19T13:15:13.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-snaps/0/keyring 2026-02-19T13:15:13.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:15:13.211 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:15:13.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:15:13.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-snaps/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:15:13.414 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:15:13.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:15:13.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:13.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:15:13.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:15:13.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:15:13.429 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:13.425+0000 7f7e0f688a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:13.431 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:13.425+0000 7f7e0f688a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:13.432 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:13.429+0000 7f7e0f688a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:13.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:15:13.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:13.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:15:13.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:13.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:13.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:13.571 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:15:13.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:13.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:13.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:15:13.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:13.801 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:13.797+0000 7f7e0f688a40 -1 Falling back to public interface 2026-02-19T13:15:14.349 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:14.345+0000 7f7e0f688a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:15:14.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:14.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:14.740 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:15:14.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:14.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:14.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:15:14.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:15.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:15.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:15.901 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:15:15.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:15:15.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:15.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:15:16.060 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1020819816,v1:127.0.0.1:6803/1020819816] [v2:127.0.0.1:6804/1020819816,v1:127.0.0.1:6805/1020819816] exists,up 42f5abfb-2b06-4eeb-907a-7398feba962b 2026-02-19T13:15:16.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:16.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:16.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:16.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:186: TEST_scrub_snaps: ceph osd set noscrub 2026-02-19T13:15:16.263 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:15:16.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:187: TEST_scrub_snaps: ceph osd set nodeep-scrub 2026-02-19T13:15:16.467 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:15:16.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:190: TEST_scrub_snaps: create_pool test 1 1 2026-02-19T13:15:16.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-19T13:15:16.670 INFO:tasks.workunit.client.0.trial144.stderr:pool 'test' created 2026-02-19T13:15:16.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:15:17.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:191: TEST_scrub_snaps: wait_for_clean 2026-02-19T13:15:17.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:15:17.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:15:17.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:15:17.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:15:17.685 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:15:17.685 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:15:17.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:15:17.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:15:17.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:15:17.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:15:17.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:15:17.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:15:17.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:15:17.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:15:17.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:15:17.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:15:17.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:15:17.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:17.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:15:18.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836482 2026-02-19T13:15:18.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836482 2026-02-19T13:15:18.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836482' 2026-02-19T13:15:18.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:18.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836482 2026-02-19T13:15:18.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:18.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:15:18.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836482 2026-02-19T13:15:18.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:18.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836482 2026-02-19T13:15:18.021 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836482 2026-02-19T13:15:18.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836482' 2026-02-19T13:15:18.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:18.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836481 -lt 21474836482 2026-02-19T13:15:18.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:15:19.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:15:19.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:19.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836482 2026-02-19T13:15:19.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:15:19.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:19.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:19.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:15:19.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:15:19.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:15:19.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:15:19.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:15:19.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:15:19.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:15:19.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:15:19.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:15:19.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:15:19.700 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:19.700 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:19.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:15:19.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:15:19.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:15:19.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:192: TEST_scrub_snaps: ceph osd dump 2026-02-19T13:15:19.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:192: TEST_scrub_snaps: grep '^pool.*['\'']test['\'']' 2026-02-19T13:15:19.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:192: TEST_scrub_snaps: awk '{ print $2 }' 2026-02-19T13:15:20.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:192: TEST_scrub_snaps: poolid=1 2026-02-19T13:15:20.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:194: TEST_scrub_snaps: dd if=/dev/urandom of=testdata.456002 bs=1032 count=1 2026-02-19T13:15:20.048 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:15:20.048 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:15:20.048 INFO:tasks.workunit.client.0.trial144.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 5.7847e-05 s, 17.8 MB/s 2026-02-19T13:15:20.048 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: seq 1 16 2026-02-19T13:15:20.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj1 testdata.456002 2026-02-19T13:15:20.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj2 testdata.456002 2026-02-19T13:15:20.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj3 testdata.456002 2026-02-19T13:15:20.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj4 testdata.456002 2026-02-19T13:15:20.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj5 testdata.456002 2026-02-19T13:15:20.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj6 testdata.456002 2026-02-19T13:15:20.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj7 testdata.456002 2026-02-19T13:15:20.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj8 testdata.456002 2026-02-19T13:15:20.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj9 testdata.456002 2026-02-19T13:15:20.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj10 testdata.456002 2026-02-19T13:15:20.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj11 testdata.456002 2026-02-19T13:15:20.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj12 testdata.456002 2026-02-19T13:15:20.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj13 testdata.456002 2026-02-19T13:15:20.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj14 testdata.456002 2026-02-19T13:15:20.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj15 testdata.456002 2026-02-19T13:15:20.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-19T13:15:20.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj16 testdata.456002 2026-02-19T13:15:20.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:200: TEST_scrub_snaps: get_primary test obj1 2026-02-19T13:15:20.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=test 2026-02-19T13:15:20.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T13:15:20.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map test obj1 2026-02-19T13:15:20.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:15:20.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:200: TEST_scrub_snaps: local primary=0 2026-02-19T13:15:20.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:202: TEST_scrub_snaps: create_scenario td/osd-scrub-snaps test testdata.456002 0 2026-02-19T13:15:20.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:45: create_scenario: local dir=td/osd-scrub-snaps 2026-02-19T13:15:20.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:46: create_scenario: local poolname=test 2026-02-19T13:15:20.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:47: create_scenario: local TESTDATA=testdata.456002 2026-02-19T13:15:20.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:48: create_scenario: local osd=0 2026-02-19T13:15:20.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:50: create_scenario: SNAP=1 2026-02-19T13:15:20.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:51: create_scenario: rados -p test mksnap snap1 2026-02-19T13:15:20.638 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap1 2026-02-19T13:15:20.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:52: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=1 2026-02-19T13:15:20.640 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:15:20.640 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:15:20.640 INFO:tasks.workunit.client.0.trial144.stderr:256 bytes copied, 5.2036e-05 s, 4.9 MB/s 2026-02-19T13:15:20.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:53: create_scenario: rados -p test put obj1 testdata.456002 2026-02-19T13:15:20.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:54: create_scenario: rados -p test put obj5 testdata.456002 2026-02-19T13:15:20.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:55: create_scenario: rados -p test put obj3 testdata.456002 2026-02-19T13:15:20.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: seq 6 14 2026-02-19T13:15:20.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:15:20.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj6 testdata.456002 2026-02-19T13:15:20.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:15:20.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj7 testdata.456002 2026-02-19T13:15:20.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:15:20.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj8 testdata.456002 2026-02-19T13:15:20.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:15:20.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj9 testdata.456002 2026-02-19T13:15:20.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:15:20.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj10 testdata.456002 2026-02-19T13:15:20.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:15:20.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj11 testdata.456002 2026-02-19T13:15:20.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:15:20.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj12 testdata.456002 2026-02-19T13:15:20.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:15:20.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj13 testdata.456002 2026-02-19T13:15:20.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:15:20.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj14 testdata.456002 2026-02-19T13:15:20.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:60: create_scenario: SNAP=2 2026-02-19T13:15:20.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:61: create_scenario: rados -p test mksnap snap2 2026-02-19T13:15:20.940 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap2 2026-02-19T13:15:20.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:62: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=2 2026-02-19T13:15:20.943 INFO:tasks.workunit.client.0.trial144.stderr:2+0 records in 2026-02-19T13:15:20.943 INFO:tasks.workunit.client.0.trial144.stderr:2+0 records out 2026-02-19T13:15:20.943 INFO:tasks.workunit.client.0.trial144.stderr:512 bytes copied, 8.5819e-05 s, 6.0 MB/s 2026-02-19T13:15:20.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:63: create_scenario: rados -p test put obj5 testdata.456002 2026-02-19T13:15:20.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:65: create_scenario: SNAP=3 2026-02-19T13:15:20.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:66: create_scenario: rados -p test mksnap snap3 2026-02-19T13:15:21.040 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap3 2026-02-19T13:15:21.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:67: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=3 2026-02-19T13:15:21.043 INFO:tasks.workunit.client.0.trial144.stderr:3+0 records in 2026-02-19T13:15:21.043 INFO:tasks.workunit.client.0.trial144.stderr:3+0 records out 2026-02-19T13:15:21.043 INFO:tasks.workunit.client.0.trial144.stderr:768 bytes copied, 5.1275e-05 s, 15.0 MB/s 2026-02-19T13:15:21.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:68: create_scenario: rados -p test put obj3 testdata.456002 2026-02-19T13:15:21.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:70: create_scenario: SNAP=4 2026-02-19T13:15:21.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:71: create_scenario: rados -p test mksnap snap4 2026-02-19T13:15:21.144 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap4 2026-02-19T13:15:21.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:72: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=4 2026-02-19T13:15:21.147 INFO:tasks.workunit.client.0.trial144.stderr:4+0 records in 2026-02-19T13:15:21.147 INFO:tasks.workunit.client.0.trial144.stderr:4+0 records out 2026-02-19T13:15:21.147 INFO:tasks.workunit.client.0.trial144.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 8.9225e-05 s, 11.5 MB/s 2026-02-19T13:15:21.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:73: create_scenario: rados -p test put obj5 testdata.456002 2026-02-19T13:15:21.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:74: create_scenario: rados -p test put obj2 testdata.456002 2026-02-19T13:15:21.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:76: create_scenario: SNAP=5 2026-02-19T13:15:21.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:77: create_scenario: rados -p test mksnap snap5 2026-02-19T13:15:21.246 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap5 2026-02-19T13:15:21.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:78: create_scenario: SNAP=6 2026-02-19T13:15:21.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:79: create_scenario: rados -p test mksnap snap6 2026-02-19T13:15:21.347 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap6 2026-02-19T13:15:21.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:80: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=6 2026-02-19T13:15:21.349 INFO:tasks.workunit.client.0.trial144.stderr:6+0 records in 2026-02-19T13:15:21.349 INFO:tasks.workunit.client.0.trial144.stderr:6+0 records out 2026-02-19T13:15:21.349 INFO:tasks.workunit.client.0.trial144.stderr:1536 bytes (1.5 kB, 1.5 KiB) copied, 0.000100767 s, 15.2 MB/s 2026-02-19T13:15:21.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:81: create_scenario: rados -p test put obj5 testdata.456002 2026-02-19T13:15:21.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:83: create_scenario: SNAP=7 2026-02-19T13:15:21.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:84: create_scenario: rados -p test mksnap snap7 2026-02-19T13:15:21.450 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap7 2026-02-19T13:15:21.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:86: create_scenario: rados -p test rm obj4 2026-02-19T13:15:21.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:87: create_scenario: rados -p test rm obj16 2026-02-19T13:15:21.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:88: create_scenario: rados -p test rm obj2 2026-02-19T13:15:21.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:90: create_scenario: kill_daemons td/osd-scrub-snaps TERM osd 2026-02-19T13:15:21.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:15:21.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:15:21.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:15:21.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:15:21.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:15:21.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:15:21.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj1 2026-02-19T13:15:21.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: JSON='["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:21.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:95: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' --force remove 2026-02-19T13:15:22.191 INFO:tasks.workunit.client.0.trial144.stdout:WARNING: only removing head with clones present 2026-02-19T13:15:22.191 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-19T13:15:22.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-19T13:15:22.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: grep '"snapid":2' 2026-02-19T13:15:22.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:22.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:98: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-19T13:15:22.610 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:c52c9666:::obj5:2# 2026-02-19T13:15:22.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-19T13:15:22.693 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: grep '"snapid":1' 2026-02-19T13:15:22.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:22.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:101: create_scenario: OBJ5SAVE='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:22.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:103: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-19T13:15:23.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:104: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-19T13:15:23.076 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:15:23.076 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-19T13:15:23.076 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:23.076 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:15:23.076 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-19T13:15:23.077 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-19T13:15:23.077 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-19T13:15:23.077 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-19T13:15:23.077 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-19T13:15:23.077 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-19T13:15:23.077 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:23.077 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-19T13:15:23.077 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-19T13:15:23.077 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-19T13:15:23.077 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-19T13:15:23.077 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:15:23.078 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:15:23.078 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:23.078 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:23.078 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:15:23.078 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:15:23.078 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:23.078 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:15:23.078 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:23.078 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:15:23.078 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:23.078 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:15:23.079 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:23.079 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:15:23.079 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:23.079 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:15:23.079 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:15:23.079 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:15:23.079 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:23.079 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:15:23.079 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:15:23.079 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:15:23.079 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:23.080 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:15:23.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:105: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-19T13:15:23.080 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:15:23.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:106: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --rmtype nosnapmap '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-19T13:15:23.249 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:c52c9666:::obj5:1# 2026-02-19T13:15:23.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:108: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-19T13:15:23.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:109: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-19T13:15:23.572 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:15:23.572 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-19T13:15:23.572 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:23.572 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:15:23.572 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-19T13:15:23.572 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-19T13:15:23.573 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-19T13:15:23.573 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-19T13:15:23.573 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-19T13:15:23.573 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-19T13:15:23.573 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:23.573 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-19T13:15:23.573 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-19T13:15:23.573 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-19T13:15:23.573 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-19T13:15:23.573 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:15:23.574 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:15:23.574 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:23.574 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:23.574 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:15:23.574 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:15:23.574 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:23.574 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:15:23.574 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:23.574 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:15:23.574 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:23.574 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:15:23.575 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:23.575 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:15:23.575 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:23.575 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:15:23.575 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:15:23.575 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:15:23.575 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:23.575 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:15:23.575 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:15:23.575 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:15:23.575 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:23.575 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:15:23.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:110: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-19T13:15:23.576 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:15:23.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:111: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-19T13:15:23.577 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-19T13:15:23.577 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: grep '"snapid":4' 2026-02-19T13:15:23.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:23.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:114: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=18 2026-02-19T13:15:23.724 INFO:tasks.workunit.client.0.trial144.stderr:18+0 records in 2026-02-19T13:15:23.724 INFO:tasks.workunit.client.0.trial144.stderr:18+0 records out 2026-02-19T13:15:23.725 INFO:tasks.workunit.client.0.trial144.stderr:4608 bytes (4.6 kB, 4.5 KiB) copied, 0.000220849 s, 20.9 MB/s 2026-02-19T13:15:23.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:115: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.456002 2026-02-19T13:15:24.008 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj3 2026-02-19T13:15:24.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: JSON='["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:24.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:118: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=15 2026-02-19T13:15:24.188 INFO:tasks.workunit.client.0.trial144.stderr:15+0 records in 2026-02-19T13:15:24.188 INFO:tasks.workunit.client.0.trial144.stderr:15+0 records out 2026-02-19T13:15:24.188 INFO:tasks.workunit.client.0.trial144.stderr:3840 bytes (3.8 kB, 3.8 KiB) copied, 0.000147303 s, 26.1 MB/s 2026-02-19T13:15:24.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:119: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.456002 2026-02-19T13:15:24.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj4 2026-02-19T13:15:24.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: grep '"snapid":7' 2026-02-19T13:15:24.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: JSON='["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:24.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:122: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-19T13:15:24.807 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:0ee9ae15:::obj4:7# 2026-02-19T13:15:24.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:125: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-19T13:15:25.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:126: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-19T13:15:25.180 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-19T13:15:25.180 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:25.180 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:15:25.180 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-19T13:15:25.180 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-19T13:15:25.180 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-19T13:15:25.180 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-19T13:15:25.180 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-19T13:15:25.180 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-19T13:15:25.180 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:25.181 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-19T13:15:25.181 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-19T13:15:25.181 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-19T13:15:25.181 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-19T13:15:25.181 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:15:25.181 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:25.181 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:25.181 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:15:25.181 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:25.181 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:15:25.181 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:25.181 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:25.181 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:15:25.182 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:25.182 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:25.182 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:15:25.182 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:15:25.182 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:25.182 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:15:25.182 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:15:25.182 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:25.182 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:15:25.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:127: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-19T13:15:25.183 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:25.183 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:25.183 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:25.183 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:25.183 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:25.183 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:25.184 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:15:25.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj16 2026-02-19T13:15:25.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: grep '"snapid":7' 2026-02-19T13:15:25.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: JSON='["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:25.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:129: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --rmtype snapmap '["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-19T13:15:25.527 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:461f8b5e:::obj16:7# 2026-02-19T13:15:25.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:131: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-19T13:15:25.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:132: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-19T13:15:25.828 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-19T13:15:25.828 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:15:25.828 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-19T13:15:25.828 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-19T13:15:25.828 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-19T13:15:25.828 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-19T13:15:25.828 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-19T13:15:25.829 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-19T13:15:25.829 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:25.829 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-19T13:15:25.829 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-19T13:15:25.829 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-19T13:15:25.829 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-19T13:15:25.829 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:15:25.829 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:25.829 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:15:25.829 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:15:25.829 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:25.829 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:15:25.830 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:15:25.830 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:15:25.830 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:15:25.830 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:15:25.830 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:15:25.830 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:15:25.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:133: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-19T13:15:25.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:134: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-19T13:15:25.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj2 2026-02-19T13:15:25.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: JSON='["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:25.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:137: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' rm-attr snapset 2026-02-19T13:15:26.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: echo '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:26.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: sed 's/snapid":1/snapid":7/' 2026-02-19T13:15:26.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:26.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:141: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=7 2026-02-19T13:15:26.303 INFO:tasks.workunit.client.0.trial144.stderr:7+0 records in 2026-02-19T13:15:26.303 INFO:tasks.workunit.client.0.trial144.stderr:7+0 records out 2026-02-19T13:15:26.303 INFO:tasks.workunit.client.0.trial144.stderr:1792 bytes (1.8 kB, 1.8 KiB) copied, 0.000187568 s, 9.6 MB/s 2026-02-19T13:15:26.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:142: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.456002 2026-02-19T13:15:26.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj6 2026-02-19T13:15:26.646 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:15:26.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: JSON='["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:26.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:145: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset 2026-02-19T13:15:27.008 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj7 2026-02-19T13:15:27.074 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:15:27.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: JSON='["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:27.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:147: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset corrupt 2026-02-19T13:15:27.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj8 2026-02-19T13:15:27.502 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:15:27.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: JSON='["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:27.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:149: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset seq 2026-02-19T13:15:27.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj9 2026-02-19T13:15:27.906 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:15:28.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: JSON='["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:28.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:151: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_size 2026-02-19T13:15:28.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj10 2026-02-19T13:15:28.354 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:15:28.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: JSON='["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:28.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:153: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_overlap 2026-02-19T13:15:28.700 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj11 2026-02-19T13:15:28.762 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:15:28.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: JSON='["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:28.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:155: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clones 2026-02-19T13:15:29.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj12 2026-02-19T13:15:29.206 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:15:29.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: JSON='["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:29.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:157: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset head 2026-02-19T13:15:29.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj13 2026-02-19T13:15:29.642 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:15:29.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: JSON='["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:29.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:159: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset snaps 2026-02-19T13:15:29.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj14 2026-02-19T13:15:30.054 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:15:30.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: JSON='["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:30.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:161: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset size 2026-02-19T13:15:30.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:163: create_scenario: echo garbage 2026-02-19T13:15:30.444 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj15 2026-02-19T13:15:30.518 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:15:30.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: JSON='["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:15:30.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:165: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' set-attr snapset td/osd-scrub-snaps/bad 2026-02-19T13:15:30.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:166: create_scenario: rm -f td/osd-scrub-snaps/bad 2026-02-19T13:15:30.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:167: create_scenario: return 0 2026-02-19T13:15:30.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:204: TEST_scrub_snaps: rm -f testdata.456002 2026-02-19T13:15:30.881 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:206: TEST_scrub_snaps: expr 1 - 1 2026-02-19T13:15:30.882 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:206: TEST_scrub_snaps: seq 0 0 2026-02-19T13:15:30.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:206: TEST_scrub_snaps: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:15:30.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:208: TEST_scrub_snaps: activate_osd td/osd-scrub-snaps 0 2026-02-19T13:15:30.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-snaps 2026-02-19T13:15:30.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:15:30.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:15:30.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:15:30.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-19T13:15:30.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-19T13:15:30.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:30.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:30.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:30.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-19T13:15:30.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-19T13:15:30.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:30.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:15:30.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-19T13:15:30.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:15:30.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:30.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:30.887 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:30.887 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:30.887 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:30.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:15:30.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:15:30.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:30.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-19T13:15:30.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-19T13:15:30.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:30.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:30.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:30.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:30.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:15:30.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:15:30.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-19T13:15:30.888 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:15:30.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:15:30.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:30.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-snaps/0/whoami 2026-02-19T13:15:30.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:15:30.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:15:30.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:15:30.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:15:30.904 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:30.900+0000 7f7d7ea8fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:30.905 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:30.900+0000 7f7d7ea8fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:30.906 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:30.900+0000 7f7d7ea8fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:31.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:15:31.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:31.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:15:31.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:31.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:31.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:31.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:31.048 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:15:31.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:31.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:15:31.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:31.305 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:31.300+0000 7f7d7ea8fa40 -1 Falling back to public interface 2026-02-19T13:15:31.912 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:31.908+0000 7f7d7ea8fa40 -1 osd.0 17 log_to_monitors true 2026-02-19T13:15:32.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:32.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:32.204 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:15:32.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:32.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:32.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:15:32.359 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 20 up_thru 20 down_at 18 last_clean_interval [5,17) [v2:127.0.0.1:6802/2559272361,v1:127.0.0.1:6803/2559272361] [v2:127.0.0.1:6804/2559272361,v1:127.0.0.1:6805/2559272361] exists,up 42f5abfb-2b06-4eeb-907a-7398feba962b 2026-02-19T13:15:32.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:32.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:32.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:32.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:210: TEST_scrub_snaps: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_max 25 2026-02-19T13:15:32.421 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:15:32.422 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_shallow_scrub_chunk_max = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-19T13:15:32.422 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:15:32.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:211: TEST_scrub_snaps: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_min 5 2026-02-19T13:15:32.501 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:15:32.502 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_min = '' " 2026-02-19T13:15:32.502 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:15:32.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:212: TEST_scrub_snaps: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_seconds 1 2026-02-19T13:15:32.581 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:15:32.581 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_pg_stat_report_interval_max_seconds = '' " 2026-02-19T13:15:32.581 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:15:32.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:213: TEST_scrub_snaps: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_epochs 1 2026-02-19T13:15:32.654 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:15:32.654 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_pg_stat_report_interval_max_epochs = '' " 2026-02-19T13:15:32.654 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:15:32.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:216: TEST_scrub_snaps: wait_for_clean 2026-02-19T13:15:32.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:15:32.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:15:32.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:15:32.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:15:32.663 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:15:32.663 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:15:32.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:15:32.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:15:32.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:15:32.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:15:32.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:15:32.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:15:32.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:15:32.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:15:32.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:15:32.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:15:32.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:15:32.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:32.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:15:32.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=85899345922 2026-02-19T13:15:32.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 85899345922 2026-02-19T13:15:32.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-85899345922' 2026-02-19T13:15:32.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:32.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-85899345922 2026-02-19T13:15:32.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:32.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:15:33.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-85899345922 2026-02-19T13:15:33.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:33.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=85899345922 2026-02-19T13:15:33.002 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 85899345922 2026-02-19T13:15:33.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 85899345922' 2026-02-19T13:15:33.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:33.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 85899345922 2026-02-19T13:15:33.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:15:34.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:15:34.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:34.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 85899345922 2026-02-19T13:15:34.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:15:35.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:15:35.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:35.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 85899345922 -lt 85899345922 2026-02-19T13:15:35.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:15:35.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:35.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:35.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:15:35.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:15:35.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:15:35.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:15:35.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:15:35.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:15:35.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:15:35.688 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:15:35.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:15:35.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:15:35.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:35.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:36.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:15:36.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:15:36.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:15:36.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:218: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_shallow_scrub_chunk_max 2026-02-19T13:15:36.102 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:15:36.103 INFO:tasks.workunit.client.0.trial144.stdout: "osd_shallow_scrub_chunk_max": "25" 2026-02-19T13:15:36.103 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:15:36.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:219: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_shallow_scrub_chunk_min 2026-02-19T13:15:36.181 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:15:36.182 INFO:tasks.workunit.client.0.trial144.stdout: "osd_shallow_scrub_chunk_min": "5" 2026-02-19T13:15:36.182 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:15:36.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:220: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_pg_stat_report_interval_max_seconds 2026-02-19T13:15:36.261 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:15:36.261 INFO:tasks.workunit.client.0.trial144.stdout: "osd_pg_stat_report_interval_max_seconds": "1" 2026-02-19T13:15:36.261 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:15:36.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:221: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_pg_stat_report_interval_max_epochs 2026-02-19T13:15:36.334 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:15:36.334 INFO:tasks.workunit.client.0.trial144.stdout: "osd_pg_stat_report_interval_max_epochs": "1" 2026-02-19T13:15:36.334 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:15:36.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:222: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_scrub_chunk_max 2026-02-19T13:15:36.407 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:15:36.407 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_chunk_max": "15" 2026-02-19T13:15:36.407 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:15:36.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:223: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_scrub_chunk_min 2026-02-19T13:15:36.478 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:15:36.478 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_chunk_min": "5" 2026-02-19T13:15:36.478 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:15:36.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:225: TEST_scrub_snaps: local pgid=1.0 2026-02-19T13:15:36.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:226: TEST_scrub_snaps: pg_scrub 1.0 2026-02-19T13:15:36.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-19T13:15:36.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-19T13:15:36.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-19T13:15:36.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:15:36.487 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:15:36.487 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:15:36.487 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:15:36.487 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:15:36.487 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:15:36.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:15:36.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:15:36.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:15:36.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:15:36.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:15:36.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:15:36.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-19T13:15:36.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:15:36.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:36.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:15:36.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=85899345923 2026-02-19T13:15:36.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 85899345923 2026-02-19T13:15:36.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-85899345923' 2026-02-19T13:15:36.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:36.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-85899345923 2026-02-19T13:15:36.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:36.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:15:36.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-85899345923 2026-02-19T13:15:36.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:36.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=85899345923 2026-02-19T13:15:36.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 85899345923' 2026-02-19T13:15:36.974 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 85899345923 2026-02-19T13:15:36.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:37.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 85899345922 -lt 85899345923 2026-02-19T13:15:37.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:15:38.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:15:38.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:38.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 85899345922 -lt 85899345923 2026-02-19T13:15:38.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:15:39.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:15:39.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:39.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 85899345924 -lt 85899345923 2026-02-19T13:15:39.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:15:39.464 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 1.0 loop 0 2026-02-19T13:15:39.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-19T13:15:39.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-19T13:15:39.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-19T13:15:39.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:15:39.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-19T13:15:39.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:15:39.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-19T13:15:39.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:15:39.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:15:39.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:15:39.537 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 1.0 2026-02-19T13:15:39.537 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:15:39.537 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:15:39.538 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:15:39.538 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:15:39.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-19T13:15:16.668869+0000 2026-02-19T13:15:39.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 1.0 2026-02-19T13:15:39.834 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.0 on osd.0 to scrub 2026-02-19T13:15:39.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 1.0 2026-02-19T13:15:16.668869+0000 2026-02-19T13:15:39.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T13:15:39.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:15:16.668869+0000 2026-02-19T13:15:39.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:15:39.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:15:39.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:15:39.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:15:39.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:15:39.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:15:39.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:15:39.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:15:40.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:15:16.668869+0000 '>' 2026-02-19T13:15:16.668869+0000 2026-02-19T13:15:40.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:15:41.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:15:41.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:15:41.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:15:41.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:15:41.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:15:41.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:15:41.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:15:41.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:15:16.668869+0000 '>' 2026-02-19T13:15:16.668869+0000 2026-02-19T13:15:41.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:15:42.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:15:42.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:15:42.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:15:42.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:15:42.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:15:42.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:15:42.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:15:42.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:15:16.668869+0000 '>' 2026-02-19T13:15:16.668869+0000 2026-02-19T13:15:42.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:15:43.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:15:43.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:15:43.328 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:15:43.329 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:15:43.329 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:15:43.329 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:15:43.329 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:15:43.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:15:40.003242+0000 '>' 2026-02-19T13:15:16.668869+0000 2026-02-19T13:15:43.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:15:43.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:230: TEST_scrub_snaps: grep '_scan_snaps start' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:43.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:230: TEST_scrub_snaps: wc -l 2026-02-19T13:15:43.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:230: TEST_scrub_snaps: test 2 = 2 2026-02-19T13:15:43.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:232: TEST_scrub_snaps: rados list-inconsistent-pg test 2026-02-19T13:15:43.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:234: TEST_scrub_snaps: jq '. | length' td/osd-scrub-snaps/json 2026-02-19T13:15:43.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:234: TEST_scrub_snaps: test 1 = 1 2026-02-19T13:15:43.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:236: TEST_scrub_snaps: jq -r '.[0]' td/osd-scrub-snaps/json 2026-02-19T13:15:43.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:236: TEST_scrub_snaps: test 1.0 = 1.0 2026-02-19T13:15:43.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:238: TEST_scrub_snaps: rados list-inconsistent-obj 1.0 2026-02-19T13:15:43.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:243: TEST_scrub_snaps: jq .inconsistents 2026-02-19T13:15:43.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:243: TEST_scrub_snaps: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-19T13:15:43.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:250: TEST_scrub_snaps: jq .inconsistents td/osd-scrub-snaps/json 2026-02-19T13:15:43.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:250: TEST_scrub_snaps: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-19T13:15:43.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:251: TEST_scrub_snaps: multidiff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-19T13:15:43.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: multidiff: diff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-19T13:15:43.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:253: TEST_scrub_snaps: rados list-inconsistent-snapset 1.0 2026-02-19T13:15:43.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:255: TEST_scrub_snaps: jq .inconsistents 2026-02-19T13:15:43.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:255: TEST_scrub_snaps: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-19T13:15:43.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:634: TEST_scrub_snaps: jq .inconsistents td/osd-scrub-snaps/json 2026-02-19T13:15:43.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:634: TEST_scrub_snaps: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-19T13:15:43.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:635: TEST_scrub_snaps: multidiff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-19T13:15:43.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: multidiff: diff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-19T13:15:43.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:636: TEST_scrub_snaps: test no = yes 2026-02-19T13:15:43.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:641: TEST_scrub_snaps: test '' = yes 2026-02-19T13:15:43.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:646: TEST_scrub_snaps: find td/osd-scrub-snaps 2026-02-19T13:15:43.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:646: TEST_scrub_snaps: grep 'osd[^/]*\.pid' 2026-02-19T13:15:43.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:646: TEST_scrub_snaps: pidfiles=td/osd-scrub-snaps/osd.0.pid 2026-02-19T13:15:43.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:647: TEST_scrub_snaps: pids= 2026-02-19T13:15:43.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:648: TEST_scrub_snaps: for pidfile in ${pidfiles} 2026-02-19T13:15:43.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:650: TEST_scrub_snaps: cat td/osd-scrub-snaps/osd.0.pid 2026-02-19T13:15:43.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:650: TEST_scrub_snaps: pids+='471319 ' 2026-02-19T13:15:43.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:653: TEST_scrub_snaps: ERRORS=0 2026-02-19T13:15:43.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: seq 1 7 2026-02-19T13:15:43.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-19T13:15:43.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap1 2026-02-19T13:15:43.712 INFO:tasks.workunit.client.0.trial144.stdout:removed pool test snap snap1 2026-02-19T13:15:43.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-19T13:15:43.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap2 2026-02-19T13:15:43.813 INFO:tasks.workunit.client.0.trial144.stdout:removed pool test snap snap2 2026-02-19T13:15:43.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-19T13:15:43.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap3 2026-02-19T13:15:43.915 INFO:tasks.workunit.client.0.trial144.stdout:removed pool test snap snap3 2026-02-19T13:15:43.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-19T13:15:43.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap4 2026-02-19T13:15:44.018 INFO:tasks.workunit.client.0.trial144.stdout:removed pool test snap snap4 2026-02-19T13:15:44.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-19T13:15:44.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap5 2026-02-19T13:15:44.120 INFO:tasks.workunit.client.0.trial144.stdout:removed pool test snap snap5 2026-02-19T13:15:44.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-19T13:15:44.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap6 2026-02-19T13:15:44.222 INFO:tasks.workunit.client.0.trial144.stdout:removed pool test snap snap6 2026-02-19T13:15:44.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-19T13:15:44.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap7 2026-02-19T13:15:44.323 INFO:tasks.workunit.client.0.trial144.stdout:removed pool test snap snap7 2026-02-19T13:15:44.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:659: TEST_scrub_snaps: sleep 5 2026-02-19T13:15:49.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:660: TEST_scrub_snaps: local -i loop=0 2026-02-19T13:15:49.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:661: TEST_scrub_snaps: ceph pg dump pgs 2026-02-19T13:15:49.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:661: TEST_scrub_snaps: grep -q snaptrim 2026-02-19T13:15:49.472 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:15:49.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:663: TEST_scrub_snaps: ceph pg dump pgs 2026-02-19T13:15:49.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:663: TEST_scrub_snaps: grep -q snaptrim_error 2026-02-19T13:15:49.641 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:15:49.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:665: TEST_scrub_snaps: break 2026-02-19T13:15:49.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:674: TEST_scrub_snaps: ceph pg dump pgs 2026-02-19T13:15:49.813 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T13:15:49.813 INFO:tasks.workunit.client.0.trial144.stdout:1.0 35 0 0 0 0 23416 0 0 60 0 60 active+clean+inconsistent+snaptrim_error 2026-02-19T13:15:44.323327+0000 23'60 29:141 [0] 0 [0] 0 17'56 2026-02-19T13:15:40.003242+0000 0'0 2026-02-19T13:15:16.668869+0000 7 0 periodic scrub scheduled @ 2026-02-20T23:42:18.353317+0000 33 0 2026-02-19T13:15:49.813 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T13:15:49.813 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T13:15:49.813 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:15:49.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:676: TEST_scrub_snaps: for pid in $pids 2026-02-19T13:15:49.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:678: TEST_scrub_snaps: kill -0 471319 2026-02-19T13:15:49.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:685: TEST_scrub_snaps: kill_daemons td/osd-scrub-snaps 2026-02-19T13:15:49.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:15:49.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:15:49.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:15:49.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:15:49.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:15:55.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:687: TEST_scrub_snaps: declare -a err_strings 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:688: TEST_scrub_snaps: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj10:.* : is missing in clone_overlap' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:689: TEST_scrub_snaps: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:7 : no '\''_'\'' attr' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:690: TEST_scrub_snaps: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:7 : is an unexpected clone' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:691: TEST_scrub_snaps: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:4 : on disk size [(]4608[)] does not match object info size [(]512[)] adjusted for ondisk to [(]512[)]' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:692: TEST_scrub_snaps: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj5:head : expected clone .*:::obj5:2' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:693: TEST_scrub_snaps: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj5:head : expected clone .*:::obj5:1' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:694: TEST_scrub_snaps: err_strings[6]='log_channel[(]cluster[)] log [[]INF[]] : scrub [0-9]*[.]0 .*:::obj5:head : 2 missing clone[(]s[)]' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:695: TEST_scrub_snaps: err_strings[7]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj8:head : snaps.seq not set' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:696: TEST_scrub_snaps: err_strings[8]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj7:1 : is an unexpected clone' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:697: TEST_scrub_snaps: err_strings[9]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj3:head : on disk size [(]3840[)] does not match object info size [(]768[)] adjusted for ondisk to [(]768[)]' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:698: TEST_scrub_snaps: err_strings[10]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj6:1 : is an unexpected clone' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:699: TEST_scrub_snaps: err_strings[11]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:head : no '\''snapset'\'' attr' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:700: TEST_scrub_snaps: err_strings[12]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:7 : clone ignored due to missing snapset' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:701: TEST_scrub_snaps: err_strings[13]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:4 : clone ignored due to missing snapset' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:702: TEST_scrub_snaps: err_strings[14]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj4:head : expected clone .*:::obj4:7' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:703: TEST_scrub_snaps: err_strings[15]='log_channel[(]cluster[)] log [[]INF[]] : scrub [0-9]*[.]0 .*:::obj4:head : 1 missing clone[(]s[)]' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:704: TEST_scrub_snaps: err_strings[16]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj1:1 : is an unexpected clone' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:705: TEST_scrub_snaps: err_strings[17]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj9:1 : is missing in clone_size' 2026-02-19T13:15:55.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:706: TEST_scrub_snaps: err_strings[18]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj11:1 : is an unexpected clone' 2026-02-19T13:15:55.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:707: TEST_scrub_snaps: err_strings[19]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj14:1 : size 1032 != clone_size 1033' 2026-02-19T13:15:55.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:708: TEST_scrub_snaps: err_strings[20]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 20 errors' 2026-02-19T13:15:55.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:709: TEST_scrub_snaps: err_strings[21]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj15:head : can'\''t decode '\''snapset'\'' attr ' 2026-02-19T13:15:55.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:710: TEST_scrub_snaps: err_strings[22]='log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' 2026-02-19T13:15:55.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj10:.* : is missing in clone_overlap' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:7 : no '\''_'\'' attr' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:7 : is an unexpected clone' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:4 : on disk size [(]4608[)] does not match object info size [(]512[)] adjusted for ondisk to [(]512[)]' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj5:head : expected clone .*:::obj5:2' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj5:head : expected clone .*:::obj5:1' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]INF[]] : scrub [0-9]*[.]0 .*:::obj5:head : 2 missing clone[(]s[)]' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj8:head : snaps.seq not set' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj7:1 : is an unexpected clone' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj3:head : on disk size [(]3840[)] does not match object info size [(]768[)] adjusted for ondisk to [(]768[)]' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj6:1 : is an unexpected clone' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:head : no '\''snapset'\'' attr' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:7 : clone ignored due to missing snapset' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:4 : clone ignored due to missing snapset' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj4:head : expected clone .*:::obj4:7' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]INF[]] : scrub [0-9]*[.]0 .*:::obj4:head : 1 missing clone[(]s[)]' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj1:1 : is an unexpected clone' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj9:1 : is missing in clone_size' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj11:1 : is an unexpected clone' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj14:1 : size 1032 != clone_size 1033' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 20 errors' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj15:head : can'\''t decode '\''snapset'\'' attr ' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-19T13:15:55.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:15:55.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:721: TEST_scrub_snaps: '[' 0 '!=' 0 ']' 2026-02-19T13:15:55.189 INFO:tasks.workunit.client.0.trial144.stdout:TEST PASSED 2026-02-19T13:15:55.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:727: TEST_scrub_snaps: echo 'TEST PASSED' 2026-02-19T13:15:55.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:728: TEST_scrub_snaps: return 0 2026-02-19T13:15:55.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:40: run: teardown td/osd-scrub-snaps 2026-02-19T13:15:55.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-snaps 2026-02-19T13:15:55.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:15:55.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-19T13:15:55.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:15:55.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:15:55.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:15:55.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:15:55.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:15:55.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:15:55.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:15:55.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:15:55.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:15:55.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:15:55.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:15:55.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:15:55.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:55.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:15:55.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:15:55.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:55.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:15:55.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:15:55.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-snaps 2026-02-19T13:15:55.210 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:15:55.210 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:55.210 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:55.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.456002 2026-02-19T13:15:55.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:15:55.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:15:55.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:37: run: for func in $funcs 2026-02-19T13:15:55.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:38: run: setup td/osd-scrub-snaps 2026-02-19T13:15:55.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-snaps 2026-02-19T13:15:55.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-snaps 2026-02-19T13:15:55.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-snaps 2026-02-19T13:15:55.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:15:55.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-19T13:15:55.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:15:55.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:15:55.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:15:55.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:15:55.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:15:55.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:15:55.216 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:15:55.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:15:55.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:15:55.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:15:55.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:15:55.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:15:55.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:55.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:15:55.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:15:55.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:55.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:15:55.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:15:55.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-snaps 2026-02-19T13:15:55.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:15:55.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:55.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:55.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.456002 2026-02-19T13:15:55.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:15:55.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:15:55.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-snaps 2026-02-19T13:15:55.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:15:55.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:55.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:55.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.456002 2026-02-19T13:15:55.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:15:55.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:15:55.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:15:55.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-snaps 1' TERM HUP INT 2026-02-19T13:15:55.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:39: run: TEST_scrub_snaps_primary td/osd-scrub-snaps 2026-02-19T13:15:55.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1177: TEST_scrub_snaps_primary: local dir=td/osd-scrub-snaps 2026-02-19T13:15:55.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1178: TEST_scrub_snaps_primary: ORIG_ARGS='--fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-19T13:15:55.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1179: TEST_scrub_snaps_primary: CEPH_ARGS+=' --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-19T13:15:55.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1180: TEST_scrub_snaps_primary: _scrub_snaps_multi td/osd-scrub-snaps primary 2026-02-19T13:15:55.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:732: _scrub_snaps_multi: local dir=td/osd-scrub-snaps 2026-02-19T13:15:55.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:733: _scrub_snaps_multi: local poolname=test 2026-02-19T13:15:55.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:734: _scrub_snaps_multi: local OBJS=16 2026-02-19T13:15:55.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:735: _scrub_snaps_multi: local OSDS=2 2026-02-19T13:15:55.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:736: _scrub_snaps_multi: local which=primary 2026-02-19T13:15:55.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:738: _scrub_snaps_multi: TESTDATA=testdata.456002 2026-02-19T13:15:55.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:740: _scrub_snaps_multi: run_mon td/osd-scrub-snaps a --osd_pool_default_size=2 2026-02-19T13:15:55.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-snaps 2026-02-19T13:15:55.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:15:55.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:15:55.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:15:55.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-snaps/a 2026-02-19T13:15:55.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-snaps/a --run-dir=td/osd-scrub-snaps --osd_pool_default_size=2 2026-02-19T13:15:55.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:15:55.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:55.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:55.254 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:55.254 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:55.254 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:55.255 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:15:55.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-snaps/a '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-snaps/log --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-19T13:15:55.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:15:55.289 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:15:55.289 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:15:55.289 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:15:55.289 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:15:55.289 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:15:55.289 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:15:55.289 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:15:55.289 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:15:55.290 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:55.290 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:55.290 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:55.290 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.456002/ceph-mon.a.asok 2026-02-19T13:15:55.290 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:15:55.290 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.456002/ceph-mon.a.asok config get fsid 2026-02-19T13:15:55.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:15:55.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:15:55.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:15:55.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:15:55.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:15:55.347 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:15:55.347 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:15:55.347 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:15:55.347 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:55.347 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:55.347 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:55.347 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.456002/ceph-mon.a.asok 2026-02-19T13:15:55.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:15:55.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.456002/ceph-mon.a.asok config get mon_host 2026-02-19T13:15:55.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:741: _scrub_snaps_multi: run_mgr td/osd-scrub-snaps x 2026-02-19T13:15:55.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-snaps 2026-02-19T13:15:55.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:15:55.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:15:55.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:15:55.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-snaps/x 2026-02-19T13:15:55.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:15:55.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:15:55.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:55.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:55.505 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:55.505 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:55.505 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:55.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:15:55.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:15:55.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-snaps/x '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:15:55.530 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: expr 2 - 1 2026-02-19T13:15:55.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: seq 0 1 2026-02-19T13:15:55.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:15:55.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:744: _scrub_snaps_multi: run_osd td/osd-scrub-snaps 0 2026-02-19T13:15:55.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-snaps 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:55.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:55.534 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:55.534 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:55.534 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:55.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:15:55.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:15:55.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:55.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:55.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:55.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-19T13:15:55.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-19T13:15:55.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:55.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:55.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:55.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:55.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:55.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:55.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-19T13:15:55.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:55.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a30ad0bb-07d8-4cbc-8aed-3d3345c0e752 2026-02-19T13:15:55.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 a30ad0bb-07d8-4cbc-8aed-3d3345c0e752' 2026-02-19T13:15:55.536 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 a30ad0bb-07d8-4cbc-8aed-3d3345c0e752 2026-02-19T13:15:55.536 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:55.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQALDZdph8XFIBAAyLM8M0kY30Ij47q+SUFN0Q== 2026-02-19T13:15:55.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQALDZdph8XFIBAAyLM8M0kY30Ij47q+SUFN0Q=="}' 2026-02-19T13:15:55.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a30ad0bb-07d8-4cbc-8aed-3d3345c0e752 -i td/osd-scrub-snaps/0/new.json 2026-02-19T13:15:55.642 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:15:55.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-snaps/0/new.json 2026-02-19T13:15:55.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQALDZdph8XFIBAAyLM8M0kY30Ij47q+SUFN0Q== --osd-uuid a30ad0bb-07d8-4cbc-8aed-3d3345c0e752 2026-02-19T13:15:55.672 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:55.672+0000 7f7e16616a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:55.674 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:55.672+0000 7f7e16616a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:55.674 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:55.672+0000 7f7e16616a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:55.675 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:55.672+0000 7f7e16616a40 -1 bdev(0x5560be374c00 td/osd-scrub-snaps/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:55.675 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:55.676+0000 7f7e16616a40 -1 bluestore(td/osd-scrub-snaps/0) _read_fsid unparsable uuid 2026-02-19T13:15:56.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-snaps/0/keyring 2026-02-19T13:15:56.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:15:56.382 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:15:56.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:15:56.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-snaps/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:15:56.593 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:15:56.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:15:56.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:56.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:15:56.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:15:56.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:15:56.626 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:56.624+0000 7efc6a26da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:56.628 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:56.628+0000 7efc6a26da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:56.629 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:56.628+0000 7efc6a26da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:56.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:15:56.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:56.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:15:56.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:56.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:56.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:56.760 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:15:56.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:56.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:56.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:15:56.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:56.957 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:56.956+0000 7efc6a26da40 -1 Falling back to public interface 2026-02-19T13:15:57.590 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:57.588+0000 7efc6a26da40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:15:57.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:57.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:57.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:57.926 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:15:57.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:57.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:15:58.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:59.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:59.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:59.089 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:15:59.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:15:59.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:59.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:15:59.248 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4085607765,v1:127.0.0.1:6803/4085607765] [v2:127.0.0.1:6804/4085607765,v1:127.0.0.1:6805/4085607765] exists,up a30ad0bb-07d8-4cbc-8aed-3d3345c0e752 2026-02-19T13:15:59.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:59.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:59.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:59.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:15:59.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:744: _scrub_snaps_multi: run_osd td/osd-scrub-snaps 1 2026-02-19T13:15:59.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-snaps 2026-02-19T13:15:59.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:59.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:15:59.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:59.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-snaps/1 2026-02-19T13:15:59.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-19T13:15:59.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:59.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:59.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:59.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/1' 2026-02-19T13:15:59.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/1/journal' 2026-02-19T13:15:59.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:59.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:59.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-19T13:15:59.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:59.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:59.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:59.254 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:59.254 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:59.254 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:15:59.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:15:59.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:15:59.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:59.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:59.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:59.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-19T13:15:59.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-19T13:15:59.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:59.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:59.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:59.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:59.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:59.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:59.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-snaps/1 2026-02-19T13:15:59.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:59.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5dd387c7-fb34-4876-9f0a-529665a7bb14 2026-02-19T13:15:59.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 5dd387c7-fb34-4876-9f0a-529665a7bb14' 2026-02-19T13:15:59.256 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 5dd387c7-fb34-4876-9f0a-529665a7bb14 2026-02-19T13:15:59.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:59.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAPDZdpvFMEEBAAXw5wojguOGQPmom82FNd6A== 2026-02-19T13:15:59.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAPDZdpvFMEEBAAXw5wojguOGQPmom82FNd6A=="}' 2026-02-19T13:15:59.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5dd387c7-fb34-4876-9f0a-529665a7bb14 -i td/osd-scrub-snaps/1/new.json 2026-02-19T13:15:59.432 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:15:59.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-snaps/1/new.json 2026-02-19T13:15:59.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAPDZdpvFMEEBAAXw5wojguOGQPmom82FNd6A== --osd-uuid 5dd387c7-fb34-4876-9f0a-529665a7bb14 2026-02-19T13:15:59.462 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:59.460+0000 7f0c74beea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:59.464 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:59.464+0000 7f0c74beea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:59.465 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:59.464+0000 7f0c74beea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:59.465 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:59.464+0000 7f0c74beea40 -1 bdev(0x564309722c00 td/osd-scrub-snaps/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:59.465 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:15:59.464+0000 7f0c74beea40 -1 bluestore(td/osd-scrub-snaps/1) _read_fsid unparsable uuid 2026-02-19T13:16:00.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-snaps/1/keyring 2026-02-19T13:16:00.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:16:00.176 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:16:00.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:16:00.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-snaps/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:16:00.381 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:16:00.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:16:00.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:16:00.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:16:00.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:16:00.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:16:00.416 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:00.416+0000 7f2854c6ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:00.417 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:00.416+0000 7f2854c6ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:00.419 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:00.416+0000 7f2854c6ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:00.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:16:00.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:16:00.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:16:00.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:16:00.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:16:00.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:00.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:16:00.544 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:16:00.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:00.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:16:00.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:16:00.845 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:00.844+0000 7f2854c6ca40 -1 Falling back to public interface 2026-02-19T13:16:01.426 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:01.424+0000 7f2854c6ca40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:16:01.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:01.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:01.702 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:16:01.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:16:01.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:01.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:16:01.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:16:02.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:02.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:02.861 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:16:02.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:16:02.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:02.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:16:03.020 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3865226356,v1:127.0.0.1:6811/3865226356] [v2:127.0.0.1:6812/3865226356,v1:127.0.0.1:6813/3865226356] exists,up 5dd387c7-fb34-4876-9f0a-529665a7bb14 2026-02-19T13:16:03.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:16:03.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:16:03.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:16:03.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:748: _scrub_snaps_multi: ceph osd set noscrub 2026-02-19T13:16:03.224 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:16:03.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:749: _scrub_snaps_multi: ceph osd set nodeep-scrub 2026-02-19T13:16:03.430 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:16:03.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:752: _scrub_snaps_multi: create_pool test 1 1 2026-02-19T13:16:03.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-19T13:16:03.633 INFO:tasks.workunit.client.0.trial144.stderr:pool 'test' created 2026-02-19T13:16:03.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:16:04.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:753: _scrub_snaps_multi: wait_for_clean 2026-02-19T13:16:04.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:16:04.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:16:04.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:16:04.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:16:04.647 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:16:04.647 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:16:04.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:16:04.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:16:04.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:16:04.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:16:04.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:16:04.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:16:04.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:16:04.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:16:04.759 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:16:04.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:16:04.926 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:16:04.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:16:04.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:04.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:16:05.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T13:16:05.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T13:16:05.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T13:16:05.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:05.003 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:16:05.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705666 2026-02-19T13:16:05.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705666 2026-02-19T13:16:05.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705666' 2026-02-19T13:16:05.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:05.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T13:16:05.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:05.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:16:05.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T13:16:05.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:05.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T13:16:05.084 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T13:16:05.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T13:16:05.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:16:05.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T13:16:05.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:16:06.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:16:06.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:16:06.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-19T13:16:06.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:06.420 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705666 2026-02-19T13:16:06.420 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:06.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:16:06.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705666 2026-02-19T13:16:06.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:06.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705666 2026-02-19T13:16:06.424 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705666 2026-02-19T13:16:06.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705666' 2026-02-19T13:16:06.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:16:06.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705666 -lt 38654705666 2026-02-19T13:16:06.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:16:06.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:16:06.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:16:06.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:16:06.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:16:06.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:16:06.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:16:06.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:16:06.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:16:06.796 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:16:06.796 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:16:06.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:16:06.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:16:06.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:16:06.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:16:07.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:16:07.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:16:07.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:16:07.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: ceph osd dump 2026-02-19T13:16:07.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: grep '^pool.*['\'']test['\'']' 2026-02-19T13:16:07.153 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: awk '{ print $2 }' 2026-02-19T13:16:07.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: poolid=1 2026-02-19T13:16:07.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:756: _scrub_snaps_multi: dd if=/dev/urandom of=testdata.456002 bs=1032 count=1 2026-02-19T13:16:07.313 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:16:07.314 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:16:07.314 INFO:tasks.workunit.client.0.trial144.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.5025e-05 s, 29.5 MB/s 2026-02-19T13:16:07.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: seq 1 16 2026-02-19T13:16:07.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj1 testdata.456002 2026-02-19T13:16:07.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj2 testdata.456002 2026-02-19T13:16:07.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj3 testdata.456002 2026-02-19T13:16:07.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj4 testdata.456002 2026-02-19T13:16:07.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj5 testdata.456002 2026-02-19T13:16:07.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj6 testdata.456002 2026-02-19T13:16:07.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj7 testdata.456002 2026-02-19T13:16:07.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj8 testdata.456002 2026-02-19T13:16:07.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj9 testdata.456002 2026-02-19T13:16:07.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj10 testdata.456002 2026-02-19T13:16:07.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj11 testdata.456002 2026-02-19T13:16:07.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj12 testdata.456002 2026-02-19T13:16:07.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj13 testdata.456002 2026-02-19T13:16:07.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj14 testdata.456002 2026-02-19T13:16:07.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj15 testdata.456002 2026-02-19T13:16:07.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:07.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj16 testdata.456002 2026-02-19T13:16:07.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:762: _scrub_snaps_multi: get_primary test obj1 2026-02-19T13:16:07.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=test 2026-02-19T13:16:07.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T13:16:07.752 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map test obj1 2026-02-19T13:16:07.752 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:16:07.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:762: _scrub_snaps_multi: local primary=1 2026-02-19T13:16:07.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:763: _scrub_snaps_multi: get_not_primary test obj1 2026-02-19T13:16:07.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=test 2026-02-19T13:16:07.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=obj1 2026-02-19T13:16:07.923 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary test obj1 2026-02-19T13:16:07.924 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=test 2026-02-19T13:16:07.924 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T13:16:07.924 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map test obj1 2026-02-19T13:16:07.924 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:16:08.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T13:16:08.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map test obj1 2026-02-19T13:16:08.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T13:16:08.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:763: _scrub_snaps_multi: local replica=0 2026-02-19T13:16:08.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:765: _scrub_snaps_multi: eval create_scenario td/osd-scrub-snaps test testdata.456002 '$primary' 2026-02-19T13:16:08.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:765: _scrub_snaps_multi: create_scenario td/osd-scrub-snaps test testdata.456002 1 2026-02-19T13:16:08.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:45: create_scenario: local dir=td/osd-scrub-snaps 2026-02-19T13:16:08.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:46: create_scenario: local poolname=test 2026-02-19T13:16:08.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:47: create_scenario: local TESTDATA=testdata.456002 2026-02-19T13:16:08.255 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:48: create_scenario: local osd=1 2026-02-19T13:16:08.255 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:50: create_scenario: SNAP=1 2026-02-19T13:16:08.255 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:51: create_scenario: rados -p test mksnap snap1 2026-02-19T13:16:08.366 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap1 2026-02-19T13:16:08.369 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:52: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=1 2026-02-19T13:16:08.371 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:16:08.371 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:16:08.371 INFO:tasks.workunit.client.0.trial144.stderr:256 bytes copied, 0.0001027 s, 2.5 MB/s 2026-02-19T13:16:08.371 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:53: create_scenario: rados -p test put obj1 testdata.456002 2026-02-19T13:16:08.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:54: create_scenario: rados -p test put obj5 testdata.456002 2026-02-19T13:16:08.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:55: create_scenario: rados -p test put obj3 testdata.456002 2026-02-19T13:16:08.453 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: seq 6 14 2026-02-19T13:16:08.454 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:08.454 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj6 testdata.456002 2026-02-19T13:16:08.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:08.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj7 testdata.456002 2026-02-19T13:16:08.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:08.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj8 testdata.456002 2026-02-19T13:16:08.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:08.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj9 testdata.456002 2026-02-19T13:16:08.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:08.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj10 testdata.456002 2026-02-19T13:16:08.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:08.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj11 testdata.456002 2026-02-19T13:16:08.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:08.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj12 testdata.456002 2026-02-19T13:16:08.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:08.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj13 testdata.456002 2026-02-19T13:16:08.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:08.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj14 testdata.456002 2026-02-19T13:16:08.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:60: create_scenario: SNAP=2 2026-02-19T13:16:08.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:61: create_scenario: rados -p test mksnap snap2 2026-02-19T13:16:08.770 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap2 2026-02-19T13:16:08.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:62: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=2 2026-02-19T13:16:08.773 INFO:tasks.workunit.client.0.trial144.stderr:2+0 records in 2026-02-19T13:16:08.773 INFO:tasks.workunit.client.0.trial144.stderr:2+0 records out 2026-02-19T13:16:08.773 INFO:tasks.workunit.client.0.trial144.stderr:512 bytes copied, 9.155e-05 s, 5.6 MB/s 2026-02-19T13:16:08.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:63: create_scenario: rados -p test put obj5 testdata.456002 2026-02-19T13:16:08.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:65: create_scenario: SNAP=3 2026-02-19T13:16:08.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:66: create_scenario: rados -p test mksnap snap3 2026-02-19T13:16:08.870 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap3 2026-02-19T13:16:08.873 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:67: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=3 2026-02-19T13:16:08.874 INFO:tasks.workunit.client.0.trial144.stderr:3+0 records in 2026-02-19T13:16:08.874 INFO:tasks.workunit.client.0.trial144.stderr:3+0 records out 2026-02-19T13:16:08.874 INFO:tasks.workunit.client.0.trial144.stderr:768 bytes copied, 8.613e-05 s, 8.9 MB/s 2026-02-19T13:16:08.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:68: create_scenario: rados -p test put obj3 testdata.456002 2026-02-19T13:16:08.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:70: create_scenario: SNAP=4 2026-02-19T13:16:08.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:71: create_scenario: rados -p test mksnap snap4 2026-02-19T13:16:08.973 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap4 2026-02-19T13:16:08.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:72: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=4 2026-02-19T13:16:08.977 INFO:tasks.workunit.client.0.trial144.stderr:4+0 records in 2026-02-19T13:16:08.977 INFO:tasks.workunit.client.0.trial144.stderr:4+0 records out 2026-02-19T13:16:08.977 INFO:tasks.workunit.client.0.trial144.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 8.8604e-05 s, 11.6 MB/s 2026-02-19T13:16:08.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:73: create_scenario: rados -p test put obj5 testdata.456002 2026-02-19T13:16:09.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:74: create_scenario: rados -p test put obj2 testdata.456002 2026-02-19T13:16:09.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:76: create_scenario: SNAP=5 2026-02-19T13:16:09.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:77: create_scenario: rados -p test mksnap snap5 2026-02-19T13:16:09.075 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap5 2026-02-19T13:16:09.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:78: create_scenario: SNAP=6 2026-02-19T13:16:09.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:79: create_scenario: rados -p test mksnap snap6 2026-02-19T13:16:09.178 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap6 2026-02-19T13:16:09.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:80: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=6 2026-02-19T13:16:09.182 INFO:tasks.workunit.client.0.trial144.stderr:6+0 records in 2026-02-19T13:16:09.182 INFO:tasks.workunit.client.0.trial144.stderr:6+0 records out 2026-02-19T13:16:09.182 INFO:tasks.workunit.client.0.trial144.stderr:1536 bytes (1.5 kB, 1.5 KiB) copied, 9.5457e-05 s, 16.1 MB/s 2026-02-19T13:16:09.182 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:81: create_scenario: rados -p test put obj5 testdata.456002 2026-02-19T13:16:09.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:83: create_scenario: SNAP=7 2026-02-19T13:16:09.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:84: create_scenario: rados -p test mksnap snap7 2026-02-19T13:16:09.279 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap7 2026-02-19T13:16:09.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:86: create_scenario: rados -p test rm obj4 2026-02-19T13:16:09.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:87: create_scenario: rados -p test rm obj16 2026-02-19T13:16:09.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:88: create_scenario: rados -p test rm obj2 2026-02-19T13:16:09.355 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:90: create_scenario: kill_daemons td/osd-scrub-snaps TERM osd 2026-02-19T13:16:09.356 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:16:09.356 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:16:09.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:16:09.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:16:09.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:16:09.665 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:16:09.665 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj1 2026-02-19T13:16:09.827 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: JSON='["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:09.827 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:95: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' --force remove 2026-02-19T13:16:10.003 INFO:tasks.workunit.client.0.trial144.stdout:WARNING: only removing head with clones present 2026-02-19T13:16:10.003 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-19T13:16:10.104 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --op list obj5 2026-02-19T13:16:10.104 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: grep '"snapid":2' 2026-02-19T13:16:10.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:10.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:98: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-19T13:16:10.433 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:c52c9666:::obj5:2# 2026-02-19T13:16:10.528 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --op list obj5 2026-02-19T13:16:10.528 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: grep '"snapid":1' 2026-02-19T13:16:10.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:10.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:101: create_scenario: OBJ5SAVE='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:10.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:103: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/1 list 2026-02-19T13:16:10.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:104: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-19T13:16:10.956 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:10.956 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-19T13:16:10.956 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:10.956 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:16:10.956 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-19T13:16:10.957 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-19T13:16:10.957 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-19T13:16:10.957 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-19T13:16:10.957 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-19T13:16:10.957 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-19T13:16:10.957 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:10.957 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-19T13:16:10.957 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-19T13:16:10.957 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-19T13:16:10.957 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-19T13:16:10.957 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:10.958 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:16:10.958 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:10.958 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:10.958 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:10.958 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:16:10.958 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:10.958 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:16:10.958 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:10.958 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:10.958 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:10.958 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:16:10.958 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:10.958 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:10.959 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:10.959 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:16:10.959 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:16:10.959 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:10.959 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:10.959 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:16:10.959 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:16:10.959 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:10.959 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:10.959 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:16:10.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:105: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-19T13:16:10.960 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:16:10.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:106: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --rmtype nosnapmap '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-19T13:16:11.139 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:c52c9666:::obj5:1# 2026-02-19T13:16:11.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:108: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/1 list 2026-02-19T13:16:11.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:109: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-19T13:16:11.412 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:11.412 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-19T13:16:11.412 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:11.412 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:16:11.412 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-19T13:16:11.412 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-19T13:16:11.412 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-19T13:16:11.413 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-19T13:16:11.413 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-19T13:16:11.413 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-19T13:16:11.413 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:11.413 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-19T13:16:11.413 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-19T13:16:11.413 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-19T13:16:11.413 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-19T13:16:11.413 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:11.413 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:16:11.413 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:11.413 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:11.414 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:11.414 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:16:11.414 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:11.414 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:16:11.414 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:11.414 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:11.414 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:11.414 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:16:11.414 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:11.414 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:11.414 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:11.414 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:16:11.415 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:16:11.415 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:11.415 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:11.415 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:16:11.415 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:16:11.415 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:11.415 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:11.415 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:16:11.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:110: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-19T13:16:11.416 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:16:11.416 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:111: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-19T13:16:11.416 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --op list obj5 2026-02-19T13:16:11.417 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: grep '"snapid":4' 2026-02-19T13:16:11.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:11.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:114: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=18 2026-02-19T13:16:11.596 INFO:tasks.workunit.client.0.trial144.stderr:18+0 records in 2026-02-19T13:16:11.596 INFO:tasks.workunit.client.0.trial144.stderr:18+0 records out 2026-02-19T13:16:11.596 INFO:tasks.workunit.client.0.trial144.stderr:4608 bytes (4.6 kB, 4.5 KiB) copied, 8.8634e-05 s, 52.0 MB/s 2026-02-19T13:16:11.597 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:115: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.456002 2026-02-19T13:16:11.856 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj3 2026-02-19T13:16:12.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: JSON='["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:12.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:118: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=15 2026-02-19T13:16:12.028 INFO:tasks.workunit.client.0.trial144.stderr:15+0 records in 2026-02-19T13:16:12.028 INFO:tasks.workunit.client.0.trial144.stderr:15+0 records out 2026-02-19T13:16:12.028 INFO:tasks.workunit.client.0.trial144.stderr:3840 bytes (3.8 kB, 3.8 KiB) copied, 0.000143265 s, 26.8 MB/s 2026-02-19T13:16:12.028 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:119: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.456002 2026-02-19T13:16:12.320 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --op list obj4 2026-02-19T13:16:12.320 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: grep '"snapid":7' 2026-02-19T13:16:12.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: JSON='["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:12.480 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:122: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-19T13:16:12.650 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:0ee9ae15:::obj4:7# 2026-02-19T13:16:12.755 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:125: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/1 list 2026-02-19T13:16:12.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:126: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-19T13:16:12.972 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-19T13:16:12.972 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:12.972 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:16:12.972 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-19T13:16:12.972 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-19T13:16:12.972 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-19T13:16:12.972 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-19T13:16:12.973 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-19T13:16:12.973 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-19T13:16:12.973 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:12.973 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-19T13:16:12.973 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-19T13:16:12.973 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-19T13:16:12.973 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-19T13:16:12.973 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:16:12.973 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:12.973 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:12.973 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:16:12.974 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:12.974 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:16:12.974 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:12.974 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:12.974 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:16:12.974 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:12.974 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:12.974 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:16:12.974 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:16:12.974 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:12.974 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:16:12.975 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:16:12.975 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:12.975 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:16:12.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:127: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-19T13:16:12.975 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:12.976 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:12.976 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:12.976 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:12.976 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:12.976 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:12.976 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:12.977 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --op list obj16 2026-02-19T13:16:12.977 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: grep '"snapid":7' 2026-02-19T13:16:13.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: JSON='["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:13.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:129: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --rmtype snapmap '["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-19T13:16:13.310 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:461f8b5e:::obj16:7# 2026-02-19T13:16:13.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:131: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/1 list 2026-02-19T13:16:13.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:132: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-19T13:16:13.624 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-19T13:16:13.624 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:16:13.624 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-19T13:16:13.625 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-19T13:16:13.625 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-19T13:16:13.625 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-19T13:16:13.625 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-19T13:16:13.625 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-19T13:16:13.625 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:13.625 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-19T13:16:13.625 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-19T13:16:13.625 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-19T13:16:13.626 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-19T13:16:13.626 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:16:13.626 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:13.626 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:16:13.626 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:16:13.626 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:13.626 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:16:13.626 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:13.626 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:16:13.626 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:16:13.627 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:16:13.627 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:16:13.627 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:16:13.628 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:133: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-19T13:16:13.628 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:134: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-19T13:16:13.628 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj2 2026-02-19T13:16:13.791 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: JSON='["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:13.791 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:137: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' rm-attr snapset 2026-02-19T13:16:14.052 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: echo '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:14.053 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: sed 's/snapid":1/snapid":7/' 2026-02-19T13:16:14.054 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:14.054 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:141: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=7 2026-02-19T13:16:14.055 INFO:tasks.workunit.client.0.trial144.stderr:7+0 records in 2026-02-19T13:16:14.055 INFO:tasks.workunit.client.0.trial144.stderr:7+0 records out 2026-02-19T13:16:14.055 INFO:tasks.workunit.client.0.trial144.stderr:1792 bytes (1.8 kB, 1.8 KiB) copied, 6.6523e-05 s, 26.9 MB/s 2026-02-19T13:16:14.055 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:142: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.456002 2026-02-19T13:16:14.356 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj6 2026-02-19T13:16:14.419 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:16:14.511 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: JSON='["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:14.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:145: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset 2026-02-19T13:16:14.816 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj7 2026-02-19T13:16:14.890 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:16:14.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: JSON='["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:14.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:147: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset corrupt 2026-02-19T13:16:15.241 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj8 2026-02-19T13:16:15.314 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:16:15.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: JSON='["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:15.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:149: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset seq 2026-02-19T13:16:15.686 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj9 2026-02-19T13:16:15.762 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:16:15.842 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: JSON='["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:15.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:151: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_size 2026-02-19T13:16:16.140 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj10 2026-02-19T13:16:16.208 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:16:16.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: JSON='["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:16.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:153: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_overlap 2026-02-19T13:16:16.636 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj11 2026-02-19T13:16:16.714 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:16:16.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: JSON='["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:16.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:155: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clones 2026-02-19T13:16:17.112 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj12 2026-02-19T13:16:17.186 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:16:17.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: JSON='["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:17.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:157: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset head 2026-02-19T13:16:17.508 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj13 2026-02-19T13:16:17.578 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:16:17.671 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: JSON='["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:17.671 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:159: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset snaps 2026-02-19T13:16:17.928 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj14 2026-02-19T13:16:18.018 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:16:18.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: JSON='["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:18.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:161: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset size 2026-02-19T13:16:18.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:163: create_scenario: echo garbage 2026-02-19T13:16:18.384 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj15 2026-02-19T13:16:18.462 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:16:18.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: JSON='["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:18.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:165: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' set-attr snapset td/osd-scrub-snaps/bad 2026-02-19T13:16:18.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:166: create_scenario: rm -f td/osd-scrub-snaps/bad 2026-02-19T13:16:18.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:167: create_scenario: return 0 2026-02-19T13:16:18.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:767: _scrub_snaps_multi: rm -f testdata.456002 2026-02-19T13:16:18.846 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: expr 2 - 1 2026-02-19T13:16:18.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: seq 0 1 2026-02-19T13:16:18.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:16:18.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:771: _scrub_snaps_multi: activate_osd td/osd-scrub-snaps 0 2026-02-19T13:16:18.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-snaps 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:18.851 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:16:18.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:16:18.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:16:18.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:16:18.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-19T13:16:18.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-19T13:16:18.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:16:18.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:16:18.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:16:18.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:16:18.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:16:18.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:16:18.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-19T13:16:18.853 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:16:18.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:16:18.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:16:18.853 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-snaps/0/whoami 2026-02-19T13:16:18.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:16:18.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:16:18.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:16:18.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:16:18.876 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:18.875+0000 7ff18cf34a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:18.877 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:18.875+0000 7ff18cf34a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:18.878 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:18.875+0000 7ff18cf34a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:19.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:16:19.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:16:19.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:16:19.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:16:19.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:16:19.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:19.020 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:16:19.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:16:19.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:19.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:16:19.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:16:19.277 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:19.275+0000 7ff18cf34a40 -1 Falling back to public interface 2026-02-19T13:16:19.847 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:19.847+0000 7ff18cf34a40 -1 osd.0 21 log_to_monitors true 2026-02-19T13:16:20.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:20.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:20.183 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:16:20.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:16:20.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:20.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:16:20.347 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 24 up_thru 24 down_at 22 last_clean_interval [5,21) [v2:127.0.0.1:6802/2215016775,v1:127.0.0.1:6803/2215016775] [v2:127.0.0.1:6804/2215016775,v1:127.0.0.1:6805/2215016775] exists,up a30ad0bb-07d8-4cbc-8aed-3d3345c0e752 2026-02-19T13:16:20.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:16:20.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:16:20.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:16:20.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:16:20.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:771: _scrub_snaps_multi: activate_osd td/osd-scrub-snaps 1 2026-02-19T13:16:20.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-snaps 2026-02-19T13:16:20.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:16:20.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:16:20.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:16:20.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-snaps/1 2026-02-19T13:16:20.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-19T13:16:20.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:16:20.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:16:20.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:16:20.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/1' 2026-02-19T13:16:20.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/1/journal' 2026-02-19T13:16:20.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:16:20.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:16:20.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-19T13:16:20.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:16:20.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:16:20.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:16:20.354 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:16:20.354 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:20.354 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:16:20.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:16:20.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:16:20.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:16:20.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-19T13:16:20.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-19T13:16:20.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:16:20.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:16:20.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:16:20.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:16:20.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:16:20.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:16:20.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-snaps/1 2026-02-19T13:16:20.355 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:16:20.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:16:20.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:16:20.357 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-snaps/1/whoami 2026-02-19T13:16:20.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:16:20.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:16:20.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:16:20.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:16:20.374 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:20.371+0000 7ffba5233a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:20.375 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:20.371+0000 7ffba5233a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:20.376 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:20.375+0000 7ffba5233a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:20.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:16:20.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:16:20.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:16:20.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:16:20.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:16:20.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:20.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:16:20.520 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:16:20.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:20.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:16:20.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:16:20.749 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:20.747+0000 7ffba5233a40 -1 Falling back to public interface 2026-02-19T13:16:21.306 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:21.303+0000 7ffba5233a40 -1 osd.1 21 log_to_monitors true 2026-02-19T13:16:21.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:21.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:21.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:16:21.681 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:16:21.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:21.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:16:21.844 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 28 up_thru 28 down_at 22 last_clean_interval [9,21) [v2:127.0.0.1:6810/1089251073,v1:127.0.0.1:6811/1089251073] [v2:127.0.0.1:6812/1089251073,v1:127.0.0.1:6813/1089251073] exists,up 5dd387c7-fb34-4876-9f0a-529665a7bb14 2026-02-19T13:16:21.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:16:21.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:16:21.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:16:21.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:774: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_max 3 2026-02-19T13:16:21.918 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:16:21.918 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_shallow_scrub_chunk_max = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-19T13:16:21.918 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:16:21.926 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:16:21.926 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_shallow_scrub_chunk_max = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-19T13:16:21.926 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:16:21.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:775: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_min 3 2026-02-19T13:16:22.006 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:16:22.006 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_min = '' " 2026-02-19T13:16:22.006 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:16:22.013 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:16:22.014 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_min = '' " 2026-02-19T13:16:22.014 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:16:22.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:776: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_scrub_chunk_min 3 2026-02-19T13:16:22.095 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:16:22.096 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_chunk_min = '' " 2026-02-19T13:16:22.096 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:16:22.103 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:16:22.103 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_chunk_min = '' " 2026-02-19T13:16:22.103 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:16:22.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:777: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_seconds 1 2026-02-19T13:16:22.191 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:16:22.191 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_pg_stat_report_interval_max_seconds = '' " 2026-02-19T13:16:22.191 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:16:22.198 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:16:22.198 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_pg_stat_report_interval_max_seconds = '' " 2026-02-19T13:16:22.199 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:16:22.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:778: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_epochs 1 2026-02-19T13:16:22.284 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:16:22.284 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_pg_stat_report_interval_max_epochs = '' " 2026-02-19T13:16:22.284 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:16:22.291 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:16:22.291 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_pg_stat_report_interval_max_epochs = '' " 2026-02-19T13:16:22.291 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:16:22.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:779: _scrub_snaps_multi: wait_for_clean 2026-02-19T13:16:22.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:16:22.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:16:22.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:16:22.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:16:22.303 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:16:22.303 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:16:22.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:16:22.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:16:22.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:16:22.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:16:22.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:16:22.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:16:22.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:16:22.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:16:22.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:16:22.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:16:22.581 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:16:22.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:16:22.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:22.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:16:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215106 2026-02-19T13:16:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215106 2026-02-19T13:16:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-103079215106' 2026-02-19T13:16:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:22.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:16:22.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084290 2026-02-19T13:16:22.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084290 2026-02-19T13:16:22.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-103079215106 1-120259084290' 2026-02-19T13:16:22.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:22.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-103079215106 2026-02-19T13:16:22.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:22.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:16:22.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-103079215106 2026-02-19T13:16:22.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:22.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=103079215106 2026-02-19T13:16:22.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 103079215106' 2026-02-19T13:16:22.743 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 103079215106 2026-02-19T13:16:22.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:16:22.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 103079215106 2026-02-19T13:16:22.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:16:23.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:16:23.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:16:24.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215106 -lt 103079215106 2026-02-19T13:16:24.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:24.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-120259084290 2026-02-19T13:16:24.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:24.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:16:24.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-120259084290 2026-02-19T13:16:24.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:24.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=120259084290 2026-02-19T13:16:24.088 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 120259084290 2026-02-19T13:16:24.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 120259084290' 2026-02-19T13:16:24.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:16:24.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084290 -lt 120259084290 2026-02-19T13:16:24.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:16:24.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:16:24.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:16:24.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:16:24.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:16:24.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:16:24.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:16:24.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:16:24.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:16:24.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:16:24.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:16:24.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:16:24.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:16:24.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:16:24.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:16:24.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:16:24.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:16:24.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:16:24.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:781: _scrub_snaps_multi: local pgid=1.0 2026-02-19T13:16:24.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:782: _scrub_snaps_multi: pg_scrub 1.0 2026-02-19T13:16:24.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-19T13:16:24.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-19T13:16:24.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-19T13:16:24.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:16:24.815 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:16:24.815 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:16:24.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:16:24.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:16:24.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:16:25.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:16:25.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:16:25.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:16:25.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:16:25.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:16:25.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:16:25.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:16:25.249 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:16:25.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:16:25.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:25.249 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:16:25.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215108 2026-02-19T13:16:25.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215108 2026-02-19T13:16:25.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-103079215108' 2026-02-19T13:16:25.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:25.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:16:25.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084291 2026-02-19T13:16:25.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084291 2026-02-19T13:16:25.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-103079215108 1-120259084291' 2026-02-19T13:16:25.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:25.407 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-103079215108 2026-02-19T13:16:25.407 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:25.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:16:25.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-103079215108 2026-02-19T13:16:25.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:25.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=103079215108 2026-02-19T13:16:25.410 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 103079215108 2026-02-19T13:16:25.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 103079215108' 2026-02-19T13:16:25.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:16:25.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215106 -lt 103079215108 2026-02-19T13:16:25.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:16:26.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:16:26.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:16:26.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215108 -lt 103079215108 2026-02-19T13:16:26.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:26.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-120259084291 2026-02-19T13:16:26.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:26.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:16:26.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-120259084291 2026-02-19T13:16:26.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:26.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=120259084291 2026-02-19T13:16:26.750 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 120259084291 2026-02-19T13:16:26.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 120259084291' 2026-02-19T13:16:26.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:16:26.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084291 -lt 120259084291 2026-02-19T13:16:26.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:16:26.930 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 1.0 loop 0 2026-02-19T13:16:26.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-19T13:16:26.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-19T13:16:26.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-19T13:16:26.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:16:26.931 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-19T13:16:26.931 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:16:27.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-19T13:16:27.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:16:27.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:16:27.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:16:27.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 1.0 2026-02-19T13:16:27.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:16:27.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:16:27.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:16:27.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:16:27.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-19T13:16:03.631455+0000 2026-02-19T13:16:27.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 1.0 2026-02-19T13:16:27.328 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.0 on osd.1 to scrub 2026-02-19T13:16:27.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 1.0 2026-02-19T13:16:03.631455+0000 2026-02-19T13:16:27.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T13:16:27.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:16:03.631455+0000 2026-02-19T13:16:27.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:16:27.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:16:27.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:16:27.338 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:16:27.338 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:16:27.338 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:16:27.339 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:16:27.339 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:16:27.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:16:03.631455+0000 '>' 2026-02-19T13:16:03.631455+0000 2026-02-19T13:16:27.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:16:28.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:16:28.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:16:28.511 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:16:28.511 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:16:28.511 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:16:28.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:16:28.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:16:28.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:16:03.631455+0000 '>' 2026-02-19T13:16:03.631455+0000 2026-02-19T13:16:28.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:16:29.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:16:29.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:16:29.679 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:16:29.679 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:16:29.679 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:16:29.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:16:29.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:16:29.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:16:03.631455+0000 '>' 2026-02-19T13:16:03.631455+0000 2026-02-19T13:16:29.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:16:30.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:16:30.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:16:30.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:16:30.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:16:30.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:16:30.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:16:30.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:16:31.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:16:03.631455+0000 '>' 2026-02-19T13:16:03.631455+0000 2026-02-19T13:16:31.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:16:32.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:16:32.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:16:32.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:16:32.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:16:32.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:16:32.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:16:32.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:16:32.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:16:28.366431+0000 '>' 2026-02-19T13:16:03.631455+0000 2026-02-19T13:16:32.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:16:32.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: grep '_scan_snaps start' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:16:32.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: wc -l 2026-02-19T13:16:32.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: test 14 -gt 3 2026-02-19T13:16:32.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: grep '_scan_snaps start' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:16:32.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: wc -l 2026-02-19T13:16:32.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: test 14 -gt 3 2026-02-19T13:16:32.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:789: _scrub_snaps_multi: rados list-inconsistent-pg test 2026-02-19T13:16:32.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:791: _scrub_snaps_multi: jq '. | length' td/osd-scrub-snaps/json 2026-02-19T13:16:32.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:791: _scrub_snaps_multi: test 1 = 1 2026-02-19T13:16:32.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:793: _scrub_snaps_multi: jq -r '.[0]' td/osd-scrub-snaps/json 2026-02-19T13:16:32.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:793: _scrub_snaps_multi: test 1.0 = 1.0 2026-02-19T13:16:32.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:795: _scrub_snaps_multi: rados list-inconsistent-obj 1.0 --format=json-pretty 2026-02-19T13:16:32.251 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:16:32.252 INFO:tasks.workunit.client.0.trial144.stdout: "epoch": 28, 2026-02-19T13:16:32.252 INFO:tasks.workunit.client.0.trial144.stdout: "inconsistents": [ 2026-02-19T13:16:32.252 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.252 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:16:32.252 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj1", 2026-02-19T13:16:32.252 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:16:32.252 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:16:32.252 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:16:32.252 INFO:tasks.workunit.client.0.trial144.stdout: "version": 18 2026-02-19T13:16:32.252 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.253 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.253 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [ 2026-02-19T13:16:32.253 INFO:tasks.workunit.client.0.trial144.stdout: "missing" 2026-02-19T13:16:32.253 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.253 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:16:32.253 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:16:32.253 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj1", 2026-02-19T13:16:32.253 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:16:32.253 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:16:32.253 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 1828249343, 2026-02-19T13:16:32.253 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:16:32.253 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:16:32.254 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:16:32.254 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.254 INFO:tasks.workunit.client.0.trial144.stdout: "version": "15'18", 2026-02-19T13:16:32.254 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "14'1", 2026-02-19T13:16:32.254 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4222.0:1", 2026-02-19T13:16:32.254 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 18, 2026-02-19T13:16:32.254 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.254 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:08.397597+0000", 2026-02-19T13:16:32.254 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:08.398884+0000", 2026-02-19T13:16:32.254 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:16:32.254 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:16:32.254 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:16:32.255 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:16:32.255 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.255 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:16:32.255 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:16:32.255 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x14caafab", 2026-02-19T13:16:32.255 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:16:32.255 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:16:32.255 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:16:32.255 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:16:32.255 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:16:32.255 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:16:32.255 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.256 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:16:32.256 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:16:32.256 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.256 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:16:32.256 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.256 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:16:32.256 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:16:32.256 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.256 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256 2026-02-19T13:16:32.256 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.256 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.256 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:16:32.256 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:16:32.257 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:16:32.257 INFO:tasks.workunit.client.0.trial144.stdout: "missing" 2026-02-19T13:16:32.257 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.257 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.257 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.257 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.257 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.257 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:16:32.257 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj10", 2026-02-19T13:16:32.257 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:16:32.257 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:16:32.257 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:16:32.258 INFO:tasks.workunit.client.0.trial144.stdout: "version": 32 2026-02-19T13:16:32.258 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.258 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:16:32.258 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:16:32.258 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.258 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:16:32.258 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:16:32.258 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:16:32.258 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj10", 2026-02-19T13:16:32.258 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:16:32.258 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:16:32.258 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 718195851, 2026-02-19T13:16:32.259 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:16:32.259 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:16:32.259 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:16:32.259 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.259 INFO:tasks.workunit.client.0.trial144.stdout: "version": "15'32", 2026-02-19T13:16:32.259 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "14'10", 2026-02-19T13:16:32.259 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4243.0:1", 2026-02-19T13:16:32.259 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 32, 2026-02-19T13:16:32.259 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.259 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:08.586974+0000", 2026-02-19T13:16:32.259 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:08.587347+0000", 2026-02-19T13:16:32.259 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:16:32.259 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:16:32.260 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:16:32.260 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:16:32.260 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.260 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:16:32.260 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:16:32.260 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x14caafab", 2026-02-19T13:16:32.260 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:16:32.260 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:16:32.260 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:16:32.260 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:16:32.260 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:16:32.260 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:16:32.261 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.261 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:16:32.261 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:16:32.261 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.261 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:16:32.261 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.261 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:16:32.261 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:16:32.261 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.261 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.261 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.261 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:16:32.262 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:16:32.262 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.262 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:16:32.262 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:16:32.262 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:16:32.262 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:16:32.262 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:16:32.262 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.262 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.262 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.262 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.262 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.263 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.263 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:16:32.263 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:16:32.263 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.263 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.263 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.263 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:16:32.263 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:16:32.263 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.263 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:16:32.263 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:16:32.263 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "????", 2026-02-19T13:16:32.263 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:16:32.264 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:16:32.264 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.264 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.264 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.264 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.264 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.264 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.264 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.264 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.264 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:16:32.264 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj11", 2026-02-19T13:16:32.264 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:16:32.264 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:16:32.265 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:16:32.265 INFO:tasks.workunit.client.0.trial144.stdout: "version": 34 2026-02-19T13:16:32.265 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.265 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:16:32.265 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:16:32.265 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.265 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:16:32.265 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:16:32.265 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:16:32.265 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj11", 2026-02-19T13:16:32.265 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:16:32.265 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:16:32.266 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 693400951, 2026-02-19T13:16:32.266 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:16:32.266 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:16:32.266 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:16:32.266 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.266 INFO:tasks.workunit.client.0.trial144.stdout: "version": "15'34", 2026-02-19T13:16:32.266 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "14'11", 2026-02-19T13:16:32.266 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4246.0:1", 2026-02-19T13:16:32.266 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 34, 2026-02-19T13:16:32.266 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.266 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:08.615467+0000", 2026-02-19T13:16:32.266 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:08.615779+0000", 2026-02-19T13:16:32.267 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:16:32.267 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:16:32.267 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:16:32.267 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:16:32.267 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.267 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:16:32.267 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:16:32.267 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x14caafab", 2026-02-19T13:16:32.267 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:16:32.267 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:16:32.267 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:16:32.267 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:16:32.267 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:16:32.268 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:16:32.268 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.268 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:16:32.268 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:16:32.268 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.268 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:16:32.268 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.268 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:16:32.268 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:16:32.268 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.268 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.268 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.269 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:16:32.269 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:16:32.269 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.269 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:16:32.269 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:16:32.269 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:16:32.269 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:16:32.269 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:16:32.269 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.269 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.269 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.269 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.270 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.270 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.270 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:16:32.270 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:16:32.270 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.270 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.270 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.270 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:16:32.270 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [] 2026-02-19T13:16:32.270 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.270 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.270 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.271 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.271 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.271 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:16:32.271 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj13", 2026-02-19T13:16:32.271 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:16:32.271 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:16:32.271 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:16:32.271 INFO:tasks.workunit.client.0.trial144.stdout: "version": 38 2026-02-19T13:16:32.271 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.271 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:16:32.271 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:16:32.271 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.272 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:16:32.272 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:16:32.272 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:16:32.272 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj13", 2026-02-19T13:16:32.272 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:16:32.272 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:16:32.272 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 2087409765, 2026-02-19T13:16:32.272 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:16:32.272 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:16:32.272 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:16:32.272 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.272 INFO:tasks.workunit.client.0.trial144.stdout: "version": "15'38", 2026-02-19T13:16:32.272 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "14'13", 2026-02-19T13:16:32.273 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4252.0:1", 2026-02-19T13:16:32.273 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 38, 2026-02-19T13:16:32.273 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.273 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:08.672707+0000", 2026-02-19T13:16:32.273 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:08.673030+0000", 2026-02-19T13:16:32.273 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:16:32.273 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:16:32.273 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:16:32.273 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:16:32.273 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.273 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:16:32.273 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:16:32.274 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x14caafab", 2026-02-19T13:16:32.274 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:16:32.274 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:16:32.274 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:16:32.274 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:16:32.274 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:16:32.274 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:16:32.274 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.274 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:16:32.274 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:16:32.274 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.274 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:16:32.275 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.275 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:16:32.275 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:16:32.275 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.275 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.275 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.275 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:16:32.275 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:16:32.275 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.275 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:16:32.275 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:16:32.275 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:16:32.275 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:16:32.276 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:16:32.276 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.276 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.276 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.276 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.276 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.276 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.276 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:16:32.276 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:16:32.276 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.276 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.276 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.276 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:16:32.277 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:16:32.277 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.277 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:16:32.277 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:16:32.277 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]" 2026-02-19T13:16:32.277 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.277 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.277 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.277 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.277 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.277 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.277 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.277 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:16:32.278 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj14", 2026-02-19T13:16:32.278 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:16:32.278 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:16:32.278 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:16:32.278 INFO:tasks.workunit.client.0.trial144.stdout: "version": 40 2026-02-19T13:16:32.278 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.278 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:16:32.278 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:16:32.278 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.278 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:16:32.278 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:16:32.278 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:16:32.278 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj14", 2026-02-19T13:16:32.279 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:16:32.279 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:16:32.279 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 2484217095, 2026-02-19T13:16:32.279 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:16:32.279 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:16:32.279 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:16:32.279 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.279 INFO:tasks.workunit.client.0.trial144.stdout: "version": "15'40", 2026-02-19T13:16:32.279 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "14'14", 2026-02-19T13:16:32.279 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4255.0:1", 2026-02-19T13:16:32.279 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 40, 2026-02-19T13:16:32.279 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.280 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:08.692431+0000", 2026-02-19T13:16:32.280 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:08.692791+0000", 2026-02-19T13:16:32.280 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:16:32.280 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:16:32.280 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:16:32.280 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:16:32.280 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.280 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:16:32.280 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:16:32.281 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x14caafab", 2026-02-19T13:16:32.281 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:16:32.281 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:16:32.281 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:16:32.281 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:16:32.281 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:16:32.281 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:16:32.281 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.281 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:16:32.281 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:16:32.281 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.281 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:16:32.282 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.282 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:16:32.282 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:16:32.282 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.282 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.282 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.282 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:16:32.282 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:16:32.282 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.282 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:16:32.282 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:16:32.282 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:16:32.282 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:16:32.283 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:16:32.283 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.283 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.283 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.283 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.283 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.283 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.283 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:16:32.283 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:16:32.283 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.283 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.283 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.284 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:16:32.284 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:16:32.284 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.284 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:16:32.284 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1033, 2026-02-19T13:16:32.284 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:16:32.284 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:16:32.284 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:16:32.284 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.284 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.284 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.284 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.285 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.285 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.285 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.285 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.285 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:16:32.285 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj15", 2026-02-19T13:16:32.285 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:16:32.285 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:16:32.285 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:16:32.285 INFO:tasks.workunit.client.0.trial144.stdout: "version": 15 2026-02-19T13:16:32.285 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.285 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.285 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [ 2026-02-19T13:16:32.286 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_corrupted" 2026-02-19T13:16:32.286 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.286 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:16:32.286 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:16:32.286 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj15", 2026-02-19T13:16:32.286 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:16:32.286 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:16:32.286 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 612772309, 2026-02-19T13:16:32.286 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:16:32.286 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:16:32.286 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:16:32.286 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.286 INFO:tasks.workunit.client.0.trial144.stdout: "version": "14'15", 2026-02-19T13:16:32.287 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "0'0", 2026-02-19T13:16:32.287 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4207.0:1", 2026-02-19T13:16:32.287 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 15, 2026-02-19T13:16:32.287 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:16:32.287 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:07.719267+0000", 2026-02-19T13:16:32.287 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:07.719573+0000", 2026-02-19T13:16:32.287 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:16:32.287 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:16:32.287 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:16:32.287 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:16:32.287 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.287 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:16:32.287 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:16:32.287 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0xe4ef7967", 2026-02-19T13:16:32.288 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:16:32.288 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:16:32.288 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:16:32.288 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:16:32.288 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:16:32.288 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:16:32.288 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.288 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:16:32.288 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:16:32.288 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.288 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:16:32.288 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.288 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:16:32.288 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:16:32.288 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.289 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:16:32.289 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.289 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 0, 2026-02-19T13:16:32.289 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [] 2026-02-19T13:16:32.289 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.289 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.289 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.289 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:16:32.289 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:16:32.289 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:16:32.289 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_corrupted" 2026-02-19T13:16:32.289 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.289 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:16:32.289 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": "Z2FyYmFnZQo=" 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj2", 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: "version": 56 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [ 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_missing" 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.290 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:16:32.291 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:16:32.291 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj2", 2026-02-19T13:16:32.291 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:16:32.291 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:16:32.291 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 1058988552, 2026-02-19T13:16:32.291 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:16:32.291 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:16:32.291 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:16:32.291 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.291 INFO:tasks.workunit.client.0.trial144.stdout: "version": "21'56", 2026-02-19T13:16:32.291 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "18'48", 2026-02-19T13:16:32.291 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4297.0:1", 2026-02-19T13:16:32.291 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 56, 2026-02-19T13:16:32.291 INFO:tasks.workunit.client.0.trial144.stdout: "size": 0, 2026-02-19T13:16:32.291 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:09.352672+0000", 2026-02-19T13:16:32.292 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:09.353077+0000", 2026-02-19T13:16:32.292 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:16:32.292 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:16:32.292 INFO:tasks.workunit.client.0.trial144.stdout: "whiteout", 2026-02-19T13:16:32.292 INFO:tasks.workunit.client.0.trial144.stdout: "dirty" 2026-02-19T13:16:32.292 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.292 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:16:32.292 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:16:32.292 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0xffffffff", 2026-02-19T13:16:32.292 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:16:32.292 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:16:32.292 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:16:32.292 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:16:32.292 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: "size": 0, 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 7, 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 4, 2026-02-19T13:16:32.293 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: 4, 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: 3, 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: 2, 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 7, 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1024, 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: 7, 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: 6, 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: 5 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.294 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_missing" 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: "size": 0 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj3", 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:16:32.295 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: "version": 44 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: "size_mismatch" 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [ 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: "size_mismatch_info", 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: "obj_size_info_mismatch" 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj3", 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 1643547569, 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:16:32.296 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "version": "17'44", 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "15'22", 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4267.0:1", 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 44, 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "size": 768, 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:08.902179+0000", 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:08.903293+0000", 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x0164c3d5", 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:16:32.297 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: "size": 768 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:16:32.298 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "size_mismatch_info", 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "obj_size_info_mismatch" 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "size": 3840, 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "object_info": { 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj3", 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 1643547569, 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "version": "17'44", 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "15'22", 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4267.0:1", 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 44, 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "size": 768, 2026-02-19T13:16:32.299 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:08.902179+0000", 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:08.903293+0000", 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x0164c3d5", 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.300 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj6", 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: "version": 24 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj6", 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:16:32.301 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 2202164420, 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "version": "15'24", 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "14'6", 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4231.0:1", 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 24, 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:08.474705+0000", 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:08.475105+0000", 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x14caafab", 2026-02-19T13:16:32.302 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:16:32.303 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [] 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.304 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj7", 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "version": 26 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj7", 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 1552453721, 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:16:32.305 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "version": "15'26", 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "14'7", 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4234.0:1", 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 26, 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:08.502904+0000", 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:08.503309+0000", 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x14caafab", 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:16:32.306 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.307 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 0, 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [] 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj8", 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "version": 28 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:16:32.308 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj8", 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 2381834917, 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "version": "15'28", 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "14'8", 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4237.0:1", 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 28, 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:08.529630+0000", 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:08.530037+0000", 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x14caafab", 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:16:32.309 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.310 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 0, 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:16:32.311 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj9", 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: "version": 30 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj9", 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:16:32.312 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 3833113727, 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "version": "15'30", 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "14'9", 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4240.0:1", 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 30, 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:08.557976+0000", 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:08.558255+0000", 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x14caafab", 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:16:32.313 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:16:32.314 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: "size": "????", 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:16:32.315 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:16:32.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:797: _scrub_snaps_multi: rados list-inconsistent-snapset 1.0 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:800: _scrub_snaps_multi: '[' primary = replica ']' 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:811: _scrub_snaps_multi: scruberrors=30 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:812: _scrub_snaps_multi: jq .inconsistents 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:812: _scrub_snaps_multi: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1066: _scrub_snaps_multi: jq .inconsistents td/osd-scrub-snaps/json 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1066: _scrub_snaps_multi: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1067: _scrub_snaps_multi: multidiff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: multidiff: diff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1068: _scrub_snaps_multi: test no = yes 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1073: _scrub_snaps_multi: test '' = yes 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: find td/osd-scrub-snaps 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: grep 'osd[^/]*\.pid' 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: pidfiles='td/osd-scrub-snaps/osd.1.pid 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:td/osd-scrub-snaps/osd.0.pid' 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1079: _scrub_snaps_multi: pids= 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1080: _scrub_snaps_multi: for pidfile in ${pidfiles} 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: cat td/osd-scrub-snaps/osd.1.pid 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: pids+='490621 ' 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1080: _scrub_snaps_multi: for pidfile in ${pidfiles} 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: cat td/osd-scrub-snaps/osd.0.pid 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: pids+='490079 ' 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1085: _scrub_snaps_multi: ERRORS=0 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1089: _scrub_snaps_multi: '[' primary = primary ']' 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: seq 1 7 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-19T13:16:32.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap1 2026-02-19T13:16:32.386 INFO:tasks.workunit.client.0.trial144.stdout:removed pool test snap snap1 2026-02-19T13:16:32.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-19T13:16:32.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap2 2026-02-19T13:16:32.488 INFO:tasks.workunit.client.0.trial144.stdout:removed pool test snap snap2 2026-02-19T13:16:32.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-19T13:16:32.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap3 2026-02-19T13:16:32.590 INFO:tasks.workunit.client.0.trial144.stdout:removed pool test snap snap3 2026-02-19T13:16:32.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-19T13:16:32.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap4 2026-02-19T13:16:32.691 INFO:tasks.workunit.client.0.trial144.stdout:removed pool test snap snap4 2026-02-19T13:16:32.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-19T13:16:32.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap5 2026-02-19T13:16:32.795 INFO:tasks.workunit.client.0.trial144.stdout:removed pool test snap snap5 2026-02-19T13:16:32.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-19T13:16:32.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap6 2026-02-19T13:16:32.895 INFO:tasks.workunit.client.0.trial144.stdout:removed pool test snap snap6 2026-02-19T13:16:32.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-19T13:16:32.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap7 2026-02-19T13:16:32.997 INFO:tasks.workunit.client.0.trial144.stdout:removed pool test snap snap7 2026-02-19T13:16:33.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1095: _scrub_snaps_multi: sleep 5 2026-02-19T13:16:38.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1096: _scrub_snaps_multi: local -i loop=0 2026-02-19T13:16:38.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1097: _scrub_snaps_multi: ceph pg dump pgs 2026-02-19T13:16:38.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1097: _scrub_snaps_multi: grep -q snaptrim 2026-02-19T13:16:38.155 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:16:38.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1099: _scrub_snaps_multi: ceph pg dump pgs 2026-02-19T13:16:38.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1099: _scrub_snaps_multi: grep -q snaptrim_error 2026-02-19T13:16:38.328 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:16:38.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1101: _scrub_snaps_multi: break 2026-02-19T13:16:38.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1111: _scrub_snaps_multi: ceph pg dump pgs 2026-02-19T13:16:38.502 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T13:16:38.502 INFO:tasks.workunit.client.0.trial144.stdout:1.0 36 0 0 0 0 24448 0 0 56 0 56 active+clean+inconsistent+snaptrim_error 2026-02-19T13:16:32.998273+0000 21'56 37:136 [1,0] 1 [1,0] 1 21'56 2026-02-19T13:16:28.366431+0000 0'0 2026-02-19T13:16:03.631455+0000 7 1 periodic scrub scheduled @ 2026-02-20T23:49:02.942637+0000 33 0 2026-02-19T13:16:38.502 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T13:16:38.502 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T13:16:38.502 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:16:38.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1113: _scrub_snaps_multi: for pid in $pids 2026-02-19T13:16:38.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1115: _scrub_snaps_multi: kill -0 490621 2026-02-19T13:16:38.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1113: _scrub_snaps_multi: for pid in $pids 2026-02-19T13:16:38.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1115: _scrub_snaps_multi: kill -0 490079 2026-02-19T13:16:38.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1122: _scrub_snaps_multi: kill_daemons td/osd-scrub-snaps 2026-02-19T13:16:38.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:16:38.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:16:38.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:16:38.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:16:38.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:16:43.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:16:43.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1124: _scrub_snaps_multi: declare -a err_strings 2026-02-19T13:16:43.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1125: _scrub_snaps_multi: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj4:7 : missing' 2026-02-19T13:16:43.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1126: _scrub_snaps_multi: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj3:head : size 3840 != size 768 from auth oi' 2026-02-19T13:16:43.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1127: _scrub_snaps_multi: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:1 : missing' 2026-02-19T13:16:43.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1128: _scrub_snaps_multi: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:2 : missing' 2026-02-19T13:16:43.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1129: _scrub_snaps_multi: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj5:4 : size 4608 != size 512 from auth oi' 2026-02-19T13:16:43.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1130: _scrub_snaps_multi: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid .*:::obj5:7 : failed to pick suitable object info' 2026-02-19T13:16:43.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1131: _scrub_snaps_multi: err_strings[6]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj1:head : missing' 2026-02-19T13:16:43.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1132: _scrub_snaps_multi: err_strings[7]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 30 errors' 2026-02-19T13:16:43.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:16:43.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj4:7 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:16:43.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:16:43.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj3:head : size 3840 != size 768 from auth oi' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:16:43.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:16:43.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:1 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:16:43.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:16:43.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:2 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:16:43.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:16:43.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj5:4 : size 4608 != size 512 from auth oi' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:16:43.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:16:43.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid .*:::obj5:7 : failed to pick suitable object info' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:16:43.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:16:43.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj1:head : missing' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:16:43.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:16:43.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 30 errors' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:16:43.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1144: _scrub_snaps_multi: declare -a rep_err_strings 2026-02-19T13:16:43.849 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: eval echo '$primary' 2026-02-19T13:16:43.849 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: echo 1 2026-02-19T13:16:43.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: osd=1 2026-02-19T13:16:43.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1146: _scrub_snaps_multi: rep_err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' 2026-02-19T13:16:43.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1147: _scrub_snaps_multi: for err_string in "${rep_err_strings[@]}" 2026-02-19T13:16:43.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1149: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:16:43.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1156: _scrub_snaps_multi: '[' 0 '!=' 0 ']' 2026-02-19T13:16:43.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1162: _scrub_snaps_multi: echo 'TEST PASSED' 2026-02-19T13:16:43.852 INFO:tasks.workunit.client.0.trial144.stdout:TEST PASSED 2026-02-19T13:16:43.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1163: _scrub_snaps_multi: return 0 2026-02-19T13:16:43.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1181: TEST_scrub_snaps_primary: err=0 2026-02-19T13:16:43.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1182: TEST_scrub_snaps_primary: CEPH_ARGS='--fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-19T13:16:43.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1183: TEST_scrub_snaps_primary: return 0 2026-02-19T13:16:43.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:40: run: teardown td/osd-scrub-snaps 2026-02-19T13:16:43.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-snaps 2026-02-19T13:16:43.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:16:43.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-19T13:16:43.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:16:43.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:16:43.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:16:43.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:16:43.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:16:43.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:16:43.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:16:43.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:16:43.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:16:43.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:16:43.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:16:43.862 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:16:43.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:16:43.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:16:43.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:16:43.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:16:43.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:16:43.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:16:43.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-snaps 2026-02-19T13:16:43.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:16:43.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:43.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:16:43.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.456002 2026-02-19T13:16:43.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:16:43.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:16:43.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:37: run: for func in $funcs 2026-02-19T13:16:43.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:38: run: setup td/osd-scrub-snaps 2026-02-19T13:16:43.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-snaps 2026-02-19T13:16:43.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-snaps 2026-02-19T13:16:43.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-snaps 2026-02-19T13:16:43.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:16:43.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-19T13:16:43.881 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:16:43.881 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:16:43.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:16:43.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:16:43.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:16:43.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:16:43.883 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:16:43.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:16:43.884 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:16:43.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:16:43.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:16:43.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:16:43.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:16:43.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:16:43.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:16:43.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:16:43.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:16:43.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:16:43.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-snaps 2026-02-19T13:16:43.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:16:43.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:43.892 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:16:43.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.456002 2026-02-19T13:16:43.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:16:43.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:16:43.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-snaps 2026-02-19T13:16:43.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:16:43.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:43.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:16:43.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.456002 2026-02-19T13:16:43.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:16:43.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:16:43.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:16:43.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-snaps 1' TERM HUP INT 2026-02-19T13:16:43.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:39: run: TEST_scrub_snaps_replica td/osd-scrub-snaps 2026-02-19T13:16:43.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1167: TEST_scrub_snaps_replica: local dir=td/osd-scrub-snaps 2026-02-19T13:16:43.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1168: TEST_scrub_snaps_replica: ORIG_ARGS='--fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-19T13:16:43.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1169: TEST_scrub_snaps_replica: CEPH_ARGS+=' --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-19T13:16:43.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1170: TEST_scrub_snaps_replica: _scrub_snaps_multi td/osd-scrub-snaps replica 2026-02-19T13:16:43.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:732: _scrub_snaps_multi: local dir=td/osd-scrub-snaps 2026-02-19T13:16:43.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:733: _scrub_snaps_multi: local poolname=test 2026-02-19T13:16:43.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:734: _scrub_snaps_multi: local OBJS=16 2026-02-19T13:16:43.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:735: _scrub_snaps_multi: local OSDS=2 2026-02-19T13:16:43.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:736: _scrub_snaps_multi: local which=replica 2026-02-19T13:16:43.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:738: _scrub_snaps_multi: TESTDATA=testdata.456002 2026-02-19T13:16:43.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:740: _scrub_snaps_multi: run_mon td/osd-scrub-snaps a --osd_pool_default_size=2 2026-02-19T13:16:43.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-snaps 2026-02-19T13:16:43.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:16:43.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:16:43.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:16:43.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-snaps/a 2026-02-19T13:16:43.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-snaps/a --run-dir=td/osd-scrub-snaps --osd_pool_default_size=2 2026-02-19T13:16:43.921 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:16:43.921 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:16:43.921 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:16:43.922 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:16:43.922 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:43.922 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:16:43.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:16:43.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-snaps/a '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-snaps/log --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-19T13:16:43.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:16:43.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:16:43.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:16:43.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:16:43.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:16:43.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:16:43.957 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:16:43.958 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:16:43.958 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:16:43.958 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:16:43.958 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:43.958 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:16:43.958 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.456002/ceph-mon.a.asok 2026-02-19T13:16:43.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:16:43.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.456002/ceph-mon.a.asok config get fsid 2026-02-19T13:16:44.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:16:44.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:16:44.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:16:44.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:16:44.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:16:44.015 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:16:44.016 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:16:44.016 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:16:44.016 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:16:44.016 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:44.016 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:16:44.016 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.456002/ceph-mon.a.asok 2026-02-19T13:16:44.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:16:44.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.456002/ceph-mon.a.asok config get mon_host 2026-02-19T13:16:44.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:741: _scrub_snaps_multi: run_mgr td/osd-scrub-snaps x 2026-02-19T13:16:44.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-snaps 2026-02-19T13:16:44.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:16:44.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:16:44.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:16:44.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-snaps/x 2026-02-19T13:16:44.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:16:44.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:16:44.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:16:44.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:16:44.178 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:16:44.178 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:44.178 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:16:44.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:16:44.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:16:44.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-snaps/x '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:16:44.201 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: expr 2 - 1 2026-02-19T13:16:44.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: seq 0 1 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:744: _scrub_snaps_multi: run_osd td/osd-scrub-snaps 0 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-snaps 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-19T13:16:44.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:16:44.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-19T13:16:44.205 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:16:44.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0952cc64-ff6f-4807-8d0f-f4a6bf12761d 2026-02-19T13:16:44.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0952cc64-ff6f-4807-8d0f-f4a6bf12761d' 2026-02-19T13:16:44.207 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 0952cc64-ff6f-4807-8d0f-f4a6bf12761d 2026-02-19T13:16:44.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:16:44.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA8DZdpRv6NDRAACoSx0X3vIB6qfl2knUcpyw== 2026-02-19T13:16:44.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA8DZdpRv6NDRAACoSx0X3vIB6qfl2knUcpyw=="}' 2026-02-19T13:16:44.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0952cc64-ff6f-4807-8d0f-f4a6bf12761d -i td/osd-scrub-snaps/0/new.json 2026-02-19T13:16:44.322 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:16:44.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-snaps/0/new.json 2026-02-19T13:16:44.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA8DZdpRv6NDRAACoSx0X3vIB6qfl2knUcpyw== --osd-uuid 0952cc64-ff6f-4807-8d0f-f4a6bf12761d 2026-02-19T13:16:44.350 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:44.347+0000 7fc4a8b6fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:44.352 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:44.351+0000 7fc4a8b6fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:44.353 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:44.351+0000 7fc4a8b6fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:44.353 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:44.351+0000 7fc4a8b6fa40 -1 bdev(0x562eaee7cc00 td/osd-scrub-snaps/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:16:44.353 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:44.351+0000 7fc4a8b6fa40 -1 bluestore(td/osd-scrub-snaps/0) _read_fsid unparsable uuid 2026-02-19T13:16:45.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-snaps/0/keyring 2026-02-19T13:16:45.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:16:45.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:16:45.109 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:16:45.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-snaps/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:16:45.321 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:16:45.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:16:45.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:16:45.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:16:45.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:16:45.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:16:45.355 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:45.351+0000 7f6ff55f9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:45.356 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:45.355+0000 7f6ff55f9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:45.357 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:45.355+0000 7f6ff55f9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:45.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:16:45.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:16:45.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:16:45.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:16:45.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:16:45.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:45.485 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:16:45.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:16:45.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:45.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:16:45.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:16:45.772 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:45.771+0000 7f6ff55f9a40 -1 Falling back to public interface 2026-02-19T13:16:46.401 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:46.399+0000 7f6ff55f9a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:16:46.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:46.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:46.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:16:46.652 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:16:46.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:46.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:16:46.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:16:47.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:47.818 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:16:47.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:47.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:16:47.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:47.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:16:47.980 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1117744563,v1:127.0.0.1:6803/1117744563] [v2:127.0.0.1:6804/1117744563,v1:127.0.0.1:6805/1117744563] exists,up 0952cc64-ff6f-4807-8d0f-f4a6bf12761d 2026-02-19T13:16:47.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:16:47.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:16:47.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:16:47.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:16:47.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:744: _scrub_snaps_multi: run_osd td/osd-scrub-snaps 1 2026-02-19T13:16:47.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-snaps 2026-02-19T13:16:47.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:16:47.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:16:47.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:16:47.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-snaps/1 2026-02-19T13:16:47.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-19T13:16:47.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:16:47.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:16:47.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:16:47.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/1' 2026-02-19T13:16:47.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/1/journal' 2026-02-19T13:16:47.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:16:47.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:16:47.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-19T13:16:47.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:16:47.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:16:47.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:16:47.986 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:16:47.986 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:47.986 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:16:47.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:16:47.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:16:47.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:16:47.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:16:47.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:16:47.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-19T13:16:47.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-19T13:16:47.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:16:47.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:16:47.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:16:47.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:16:47.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:16:47.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:16:47.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-snaps/1 2026-02-19T13:16:47.987 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:16:47.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=530c2390-655b-4b76-86a0-719c8728a299 2026-02-19T13:16:47.988 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 530c2390-655b-4b76-86a0-719c8728a299 2026-02-19T13:16:47.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 530c2390-655b-4b76-86a0-719c8728a299' 2026-02-19T13:16:47.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:16:48.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA/DZdpYvWWOxAAUNqDURLsAZ4wAi+REVtg6w== 2026-02-19T13:16:48.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA/DZdpYvWWOxAAUNqDURLsAZ4wAi+REVtg6w=="}' 2026-02-19T13:16:48.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 530c2390-655b-4b76-86a0-719c8728a299 -i td/osd-scrub-snaps/1/new.json 2026-02-19T13:16:48.156 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:16:48.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-snaps/1/new.json 2026-02-19T13:16:48.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA/DZdpYvWWOxAAUNqDURLsAZ4wAi+REVtg6w== --osd-uuid 530c2390-655b-4b76-86a0-719c8728a299 2026-02-19T13:16:48.197 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:48.195+0000 7f6e56dffa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:48.199 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:48.199+0000 7f6e56dffa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:48.200 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:48.199+0000 7f6e56dffa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:48.200 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:48.199+0000 7f6e56dffa40 -1 bdev(0x55ee747aec00 td/osd-scrub-snaps/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:16:48.201 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:48.199+0000 7f6e56dffa40 -1 bluestore(td/osd-scrub-snaps/1) _read_fsid unparsable uuid 2026-02-19T13:16:48.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-snaps/1/keyring 2026-02-19T13:16:48.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:16:48.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:16:48.811 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:16:48.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-snaps/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:16:49.021 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:16:49.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:16:49.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:16:49.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:16:49.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:16:49.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:16:49.054 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:49.051+0000 7f83a1d68a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:49.056 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:49.055+0000 7f83a1d68a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:49.057 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:49.055+0000 7f83a1d68a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:49.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:16:49.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:16:49.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:16:49.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:16:49.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:16:49.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:49.188 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:16:49.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:16:49.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:49.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:16:49.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:16:49.449 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:49.447+0000 7f83a1d68a40 -1 Falling back to public interface 2026-02-19T13:16:50.025 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:16:50.023+0000 7f83a1d68a40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:16:50.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:50.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:50.359 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:16:50.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:16:50.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:50.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:16:50.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:16:51.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:51.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:51.518 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:16:51.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:16:51.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:51.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:16:51.679 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1507974767,v1:127.0.0.1:6811/1507974767] [v2:127.0.0.1:6812/1507974767,v1:127.0.0.1:6813/1507974767] exists,up 530c2390-655b-4b76-86a0-719c8728a299 2026-02-19T13:16:51.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:16:51.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:16:51.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:16:51.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:748: _scrub_snaps_multi: ceph osd set noscrub 2026-02-19T13:16:51.885 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:16:51.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:749: _scrub_snaps_multi: ceph osd set nodeep-scrub 2026-02-19T13:16:52.090 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:16:52.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:752: _scrub_snaps_multi: create_pool test 1 1 2026-02-19T13:16:52.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-19T13:16:52.293 INFO:tasks.workunit.client.0.trial144.stderr:pool 'test' created 2026-02-19T13:16:52.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:16:53.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:753: _scrub_snaps_multi: wait_for_clean 2026-02-19T13:16:53.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:16:53.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:16:53.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:16:53.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:16:53.306 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:16:53.306 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:16:53.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:16:53.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:16:53.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:16:53.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:16:53.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:16:53.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:16:53.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:16:53.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:16:53.418 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:16:53.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:16:53.584 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:16:53.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:16:53.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:53.584 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:16:53.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-19T13:16:53.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-19T13:16:53.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483' 2026-02-19T13:16:53.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:53.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:16:53.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705666 2026-02-19T13:16:53.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705666 2026-02-19T13:16:53.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836483 1-38654705666' 2026-02-19T13:16:53.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:53.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-19T13:16:53.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:53.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:16:53.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836483 2026-02-19T13:16:53.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:53.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836483 2026-02-19T13:16:53.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-19T13:16:53.742 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836483 2026-02-19T13:16:53.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:16:53.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-19T13:16:53.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:16:54.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:16:54.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:16:55.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-19T13:16:55.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:55.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705666 2026-02-19T13:16:55.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:55.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:16:55.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705666 2026-02-19T13:16:55.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:55.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705666 2026-02-19T13:16:55.083 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705666 2026-02-19T13:16:55.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705666' 2026-02-19T13:16:55.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:16:55.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705666 -lt 38654705666 2026-02-19T13:16:55.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:16:55.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:16:55.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:16:55.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:16:55.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:16:55.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:16:55.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:16:55.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:16:55.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:16:55.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:16:55.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:16:55.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:16:55.615 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:16:55.615 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:16:55.615 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:16:55.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:16:55.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:16:55.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:16:55.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: ceph osd dump 2026-02-19T13:16:55.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: grep '^pool.*['\'']test['\'']' 2026-02-19T13:16:55.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: awk '{ print $2 }' 2026-02-19T13:16:55.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: poolid=1 2026-02-19T13:16:55.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:756: _scrub_snaps_multi: dd if=/dev/urandom of=testdata.456002 bs=1032 count=1 2026-02-19T13:16:55.975 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:16:55.975 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:16:55.975 INFO:tasks.workunit.client.0.trial144.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.1518e-05 s, 32.7 MB/s 2026-02-19T13:16:55.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: seq 1 16 2026-02-19T13:16:55.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:55.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj1 testdata.456002 2026-02-19T13:16:55.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:55.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj2 testdata.456002 2026-02-19T13:16:56.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:56.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj3 testdata.456002 2026-02-19T13:16:56.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:56.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj4 testdata.456002 2026-02-19T13:16:56.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:56.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj5 testdata.456002 2026-02-19T13:16:56.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:56.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj6 testdata.456002 2026-02-19T13:16:56.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:56.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj7 testdata.456002 2026-02-19T13:16:56.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:56.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj8 testdata.456002 2026-02-19T13:16:56.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:56.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj9 testdata.456002 2026-02-19T13:16:56.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:56.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj10 testdata.456002 2026-02-19T13:16:56.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:56.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj11 testdata.456002 2026-02-19T13:16:56.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:56.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj12 testdata.456002 2026-02-19T13:16:56.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:56.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj13 testdata.456002 2026-02-19T13:16:56.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:56.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj14 testdata.456002 2026-02-19T13:16:56.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:56.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj15 testdata.456002 2026-02-19T13:16:56.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-19T13:16:56.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj16 testdata.456002 2026-02-19T13:16:56.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:762: _scrub_snaps_multi: get_primary test obj1 2026-02-19T13:16:56.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=test 2026-02-19T13:16:56.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T13:16:56.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map test obj1 2026-02-19T13:16:56.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:16:56.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:762: _scrub_snaps_multi: local primary=1 2026-02-19T13:16:56.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:763: _scrub_snaps_multi: get_not_primary test obj1 2026-02-19T13:16:56.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=test 2026-02-19T13:16:56.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=obj1 2026-02-19T13:16:56.604 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary test obj1 2026-02-19T13:16:56.604 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=test 2026-02-19T13:16:56.604 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T13:16:56.604 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map test obj1 2026-02-19T13:16:56.604 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:16:56.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T13:16:56.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map test obj1 2026-02-19T13:16:56.768 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T13:16:56.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:763: _scrub_snaps_multi: local replica=0 2026-02-19T13:16:56.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:765: _scrub_snaps_multi: eval create_scenario td/osd-scrub-snaps test testdata.456002 '$replica' 2026-02-19T13:16:56.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:765: _scrub_snaps_multi: create_scenario td/osd-scrub-snaps test testdata.456002 0 2026-02-19T13:16:56.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:45: create_scenario: local dir=td/osd-scrub-snaps 2026-02-19T13:16:56.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:46: create_scenario: local poolname=test 2026-02-19T13:16:56.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:47: create_scenario: local TESTDATA=testdata.456002 2026-02-19T13:16:56.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:48: create_scenario: local osd=0 2026-02-19T13:16:56.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:50: create_scenario: SNAP=1 2026-02-19T13:16:56.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:51: create_scenario: rados -p test mksnap snap1 2026-02-19T13:16:57.044 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap1 2026-02-19T13:16:57.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:52: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=1 2026-02-19T13:16:57.047 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:16:57.047 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:16:57.047 INFO:tasks.workunit.client.0.trial144.stderr:256 bytes copied, 6.7605e-05 s, 3.8 MB/s 2026-02-19T13:16:57.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:53: create_scenario: rados -p test put obj1 testdata.456002 2026-02-19T13:16:57.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:54: create_scenario: rados -p test put obj5 testdata.456002 2026-02-19T13:16:57.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:55: create_scenario: rados -p test put obj3 testdata.456002 2026-02-19T13:16:57.137 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: seq 6 14 2026-02-19T13:16:57.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:57.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj6 testdata.456002 2026-02-19T13:16:57.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:57.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj7 testdata.456002 2026-02-19T13:16:57.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:57.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj8 testdata.456002 2026-02-19T13:16:57.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:57.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj9 testdata.456002 2026-02-19T13:16:57.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:57.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj10 testdata.456002 2026-02-19T13:16:57.277 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:57.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj11 testdata.456002 2026-02-19T13:16:57.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:57.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj12 testdata.456002 2026-02-19T13:16:57.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:57.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj13 testdata.456002 2026-02-19T13:16:57.364 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-19T13:16:57.365 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj14 testdata.456002 2026-02-19T13:16:57.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:60: create_scenario: SNAP=2 2026-02-19T13:16:57.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:61: create_scenario: rados -p test mksnap snap2 2026-02-19T13:16:57.470 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap2 2026-02-19T13:16:57.472 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:62: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=2 2026-02-19T13:16:57.474 INFO:tasks.workunit.client.0.trial144.stderr:2+0 records in 2026-02-19T13:16:57.474 INFO:tasks.workunit.client.0.trial144.stderr:2+0 records out 2026-02-19T13:16:57.474 INFO:tasks.workunit.client.0.trial144.stderr:512 bytes copied, 9.8753e-05 s, 5.2 MB/s 2026-02-19T13:16:57.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:63: create_scenario: rados -p test put obj5 testdata.456002 2026-02-19T13:16:57.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:65: create_scenario: SNAP=3 2026-02-19T13:16:57.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:66: create_scenario: rados -p test mksnap snap3 2026-02-19T13:16:57.571 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap3 2026-02-19T13:16:57.573 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:67: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=3 2026-02-19T13:16:57.574 INFO:tasks.workunit.client.0.trial144.stderr:3+0 records in 2026-02-19T13:16:57.574 INFO:tasks.workunit.client.0.trial144.stderr:3+0 records out 2026-02-19T13:16:57.574 INFO:tasks.workunit.client.0.trial144.stderr:768 bytes copied, 9.5537e-05 s, 8.0 MB/s 2026-02-19T13:16:57.574 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:68: create_scenario: rados -p test put obj3 testdata.456002 2026-02-19T13:16:57.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:70: create_scenario: SNAP=4 2026-02-19T13:16:57.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:71: create_scenario: rados -p test mksnap snap4 2026-02-19T13:16:57.674 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap4 2026-02-19T13:16:57.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:72: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=4 2026-02-19T13:16:57.676 INFO:tasks.workunit.client.0.trial144.stderr:4+0 records in 2026-02-19T13:16:57.676 INFO:tasks.workunit.client.0.trial144.stderr:4+0 records out 2026-02-19T13:16:57.676 INFO:tasks.workunit.client.0.trial144.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 8.2353e-05 s, 12.4 MB/s 2026-02-19T13:16:57.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:73: create_scenario: rados -p test put obj5 testdata.456002 2026-02-19T13:16:57.705 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:74: create_scenario: rados -p test put obj2 testdata.456002 2026-02-19T13:16:57.734 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:76: create_scenario: SNAP=5 2026-02-19T13:16:57.734 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:77: create_scenario: rados -p test mksnap snap5 2026-02-19T13:16:57.774 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap5 2026-02-19T13:16:57.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:78: create_scenario: SNAP=6 2026-02-19T13:16:57.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:79: create_scenario: rados -p test mksnap snap6 2026-02-19T13:16:57.878 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap6 2026-02-19T13:16:57.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:80: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=6 2026-02-19T13:16:57.881 INFO:tasks.workunit.client.0.trial144.stderr:6+0 records in 2026-02-19T13:16:57.881 INFO:tasks.workunit.client.0.trial144.stderr:6+0 records out 2026-02-19T13:16:57.881 INFO:tasks.workunit.client.0.trial144.stderr:1536 bytes (1.5 kB, 1.5 KiB) copied, 0.000103682 s, 14.8 MB/s 2026-02-19T13:16:57.881 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:81: create_scenario: rados -p test put obj5 testdata.456002 2026-02-19T13:16:57.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:83: create_scenario: SNAP=7 2026-02-19T13:16:57.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:84: create_scenario: rados -p test mksnap snap7 2026-02-19T13:16:57.980 INFO:tasks.workunit.client.0.trial144.stdout:created pool test snap snap7 2026-02-19T13:16:57.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:86: create_scenario: rados -p test rm obj4 2026-02-19T13:16:58.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:87: create_scenario: rados -p test rm obj16 2026-02-19T13:16:58.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:88: create_scenario: rados -p test rm obj2 2026-02-19T13:16:58.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:90: create_scenario: kill_daemons td/osd-scrub-snaps TERM osd 2026-02-19T13:16:58.062 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:16:58.062 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:16:58.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:16:58.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:16:58.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:16:58.370 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:16:58.371 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj1 2026-02-19T13:16:58.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: JSON='["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:58.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:95: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' --force remove 2026-02-19T13:16:58.767 INFO:tasks.workunit.client.0.trial144.stdout:WARNING: only removing head with clones present 2026-02-19T13:16:58.767 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-19T13:16:58.860 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-19T13:16:58.861 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: grep '"snapid":2' 2026-02-19T13:16:59.051 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:59.052 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:98: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-19T13:16:59.225 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:c52c9666:::obj5:2# 2026-02-19T13:16:59.332 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-19T13:16:59.332 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: grep '"snapid":1' 2026-02-19T13:16:59.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:59.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:101: create_scenario: OBJ5SAVE='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:16:59.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:103: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-19T13:16:59.734 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:104: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-19T13:16:59.736 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:59.736 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-19T13:16:59.736 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:59.736 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:16:59.736 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-19T13:16:59.736 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-19T13:16:59.736 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-19T13:16:59.736 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-19T13:16:59.736 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-19T13:16:59.736 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-19T13:16:59.736 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:59.736 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:59.737 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:16:59.738 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:16:59.738 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:59.738 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:59.738 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:16:59.738 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:16:59.738 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:16:59.738 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:16:59.738 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:16:59.738 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:105: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-19T13:16:59.738 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:16:59.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:106: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --rmtype nosnapmap '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-19T13:16:59.894 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:c52c9666:::obj5:1# 2026-02-19T13:17:00.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:108: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-19T13:17:00.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:109: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-19T13:17:00.248 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:17:00.248 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-19T13:17:00.248 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:00.248 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:17:00.248 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-19T13:17:00.248 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-19T13:17:00.249 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-19T13:17:00.249 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-19T13:17:00.249 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-19T13:17:00.249 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-19T13:17:00.249 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:17:00.249 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-19T13:17:00.249 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-19T13:17:00.249 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-19T13:17:00.249 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-19T13:17:00.249 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:17:00.250 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:17:00.250 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:00.250 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:17:00.250 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:17:00.250 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:17:00.250 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:00.250 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:17:00.250 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:17:00.250 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:17:00.250 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:00.250 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:17:00.251 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:17:00.251 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:17:00.251 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:00.251 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:17:00.251 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:17:00.251 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:17:00.251 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:00.251 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:17:00.251 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:17:00.251 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-19T13:17:00.252 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:00.252 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:17:00.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:110: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-19T13:17:00.252 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:17:00.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:111: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-19T13:17:00.253 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-19T13:17:00.253 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: grep '"snapid":4' 2026-02-19T13:17:00.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:00.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:114: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=18 2026-02-19T13:17:00.424 INFO:tasks.workunit.client.0.trial144.stderr:18+0 records in 2026-02-19T13:17:00.424 INFO:tasks.workunit.client.0.trial144.stderr:18+0 records out 2026-02-19T13:17:00.424 INFO:tasks.workunit.client.0.trial144.stderr:4608 bytes (4.6 kB, 4.5 KiB) copied, 0.000165427 s, 27.9 MB/s 2026-02-19T13:17:00.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:115: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.456002 2026-02-19T13:17:00.716 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj3 2026-02-19T13:17:00.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: JSON='["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:00.911 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:118: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=15 2026-02-19T13:17:00.912 INFO:tasks.workunit.client.0.trial144.stderr:15+0 records in 2026-02-19T13:17:00.912 INFO:tasks.workunit.client.0.trial144.stderr:15+0 records out 2026-02-19T13:17:00.912 INFO:tasks.workunit.client.0.trial144.stderr:3840 bytes (3.8 kB, 3.8 KiB) copied, 0.000141793 s, 27.1 MB/s 2026-02-19T13:17:00.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:119: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.456002 2026-02-19T13:17:01.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj4 2026-02-19T13:17:01.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: grep '"snapid":7' 2026-02-19T13:17:01.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: JSON='["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:01.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:122: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-19T13:17:01.574 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:0ee9ae15:::obj4:7# 2026-02-19T13:17:01.671 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:125: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-19T13:17:01.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:126: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-19T13:17:01.912 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-19T13:17:01.912 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:01.912 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:17:01.912 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-19T13:17:01.912 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-19T13:17:01.912 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-19T13:17:01.912 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-19T13:17:01.912 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-19T13:17:01.912 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-19T13:17:01.913 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:17:01.913 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-19T13:17:01.913 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-19T13:17:01.913 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-19T13:17:01.913 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-19T13:17:01.913 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:17:01.913 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:01.913 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:17:01.913 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:17:01.913 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:01.913 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:17:01.913 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:17:01.914 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:01.914 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:17:01.914 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:17:01.914 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:01.914 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:17:01.914 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:17:01.914 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:01.914 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:17:01.914 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:17:01.914 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:01.914 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:17:01.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:127: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-19T13:17:01.915 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:01.916 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:01.916 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:01.916 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:01.916 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:01.916 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:01.916 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-19T13:17:01.916 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj16 2026-02-19T13:17:01.917 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: grep '"snapid":7' 2026-02-19T13:17:02.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: JSON='["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:02.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:129: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --rmtype snapmap '["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-19T13:17:02.278 INFO:tasks.workunit.client.0.trial144.stdout:remove #1:461f8b5e:::obj16:7# 2026-02-19T13:17:02.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:131: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-19T13:17:02.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:132: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-19T13:17:02.591 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-19T13:17:02.591 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:133: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-19T13:17:02.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:134: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-19T13:17:02.594 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj2 2026-02-19T13:17:02.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: JSON='["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:02.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:137: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' rm-attr snapset 2026-02-19T13:17:03.080 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: echo '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:03.080 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: sed 's/snapid":1/snapid":7/' 2026-02-19T13:17:03.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:03.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:141: create_scenario: dd if=/dev/urandom of=testdata.456002 bs=256 count=7 2026-02-19T13:17:03.083 INFO:tasks.workunit.client.0.trial144.stderr:7+0 records in 2026-02-19T13:17:03.083 INFO:tasks.workunit.client.0.trial144.stderr:7+0 records out 2026-02-19T13:17:03.083 INFO:tasks.workunit.client.0.trial144.stderr:1792 bytes (1.8 kB, 1.8 KiB) copied, 0.000111206 s, 16.1 MB/s 2026-02-19T13:17:03.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:142: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.456002 2026-02-19T13:17:03.322 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj6 2026-02-19T13:17:03.378 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:17:03.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: JSON='["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:03.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:145: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset 2026-02-19T13:17:03.784 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj7 2026-02-19T13:17:03.858 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:17:03.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: JSON='["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:03.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:147: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset corrupt 2026-02-19T13:17:04.248 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj8 2026-02-19T13:17:04.322 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:17:04.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: JSON='["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:04.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:149: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset seq 2026-02-19T13:17:04.700 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj9 2026-02-19T13:17:04.778 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:17:04.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: JSON='["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:04.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:151: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_size 2026-02-19T13:17:05.164 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj10 2026-02-19T13:17:05.250 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:17:05.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: JSON='["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:05.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:153: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_overlap 2026-02-19T13:17:05.612 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj11 2026-02-19T13:17:05.682 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:17:05.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: JSON='["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:05.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:155: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clones 2026-02-19T13:17:06.068 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj12 2026-02-19T13:17:06.150 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:17:06.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: JSON='["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:06.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:157: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset head 2026-02-19T13:17:06.480 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj13 2026-02-19T13:17:06.558 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:17:06.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: JSON='["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:06.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:159: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset snaps 2026-02-19T13:17:06.943 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj14 2026-02-19T13:17:07.022 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:17:07.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: JSON='["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:07.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:161: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset size 2026-02-19T13:17:07.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:163: create_scenario: echo garbage 2026-02-19T13:17:07.424 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj15 2026-02-19T13:17:07.518 INFO:tasks.workunit.client.0.trial144.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-19T13:17:07.598 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: JSON='["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:17:07.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:165: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' set-attr snapset td/osd-scrub-snaps/bad 2026-02-19T13:17:07.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:166: create_scenario: rm -f td/osd-scrub-snaps/bad 2026-02-19T13:17:07.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:167: create_scenario: return 0 2026-02-19T13:17:07.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:767: _scrub_snaps_multi: rm -f testdata.456002 2026-02-19T13:17:07.862 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: expr 2 - 1 2026-02-19T13:17:07.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: seq 0 1 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:771: _scrub_snaps_multi: activate_osd td/osd-scrub-snaps 0 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-snaps 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:17:07.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:07.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:07.866 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:07.866 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:07.866 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:17:07.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:17:07.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:17:07.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:17:07.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-19T13:17:07.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-19T13:17:07.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:17:07.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:17:07.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:17:07.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:17:07.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:17:07.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:17:07.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-19T13:17:07.868 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:17:07.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-19T13:17:07.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:17:07.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-snaps/0/whoami 2026-02-19T13:17:07.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-19T13:17:07.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:17:07.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:17:07.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:17:07.893 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:07.890+0000 7fb486588a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:07.895 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:07.890+0000 7fb486588a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:07.896 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:07.894+0000 7fb486588a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:08.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-19T13:17:08.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:17:08.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:17:08.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:17:08.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:17:08.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:08.038 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:17:08.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:17:08.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:08.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:17:08.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:17:08.289 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:08.286+0000 7fb486588a40 -1 Falling back to public interface 2026-02-19T13:17:08.887 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:08.882+0000 7fb486588a40 -1 osd.0 22 log_to_monitors true 2026-02-19T13:17:09.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:17:09.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:09.205 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:17:09.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:17:09.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:09.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:17:09.364 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 25 up_thru 25 down_at 23 last_clean_interval [5,22) [v2:127.0.0.1:6802/1144221924,v1:127.0.0.1:6803/1144221924] [v2:127.0.0.1:6804/1144221924,v1:127.0.0.1:6805/1144221924] exists,up 0952cc64-ff6f-4807-8d0f-f4a6bf12761d 2026-02-19T13:17:09.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:17:09.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:17:09.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:17:09.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:17:09.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:771: _scrub_snaps_multi: activate_osd td/osd-scrub-snaps 1 2026-02-19T13:17:09.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-snaps 2026-02-19T13:17:09.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:17:09.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:17:09.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:17:09.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-snaps/1 2026-02-19T13:17:09.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-19T13:17:09.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:17:09.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:17:09.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:17:09.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/1' 2026-02-19T13:17:09.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/1/journal' 2026-02-19T13:17:09.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:17:09.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:17:09.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-19T13:17:09.369 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:17:09.369 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:09.370 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:09.370 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:09.370 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:09.370 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:17:09.370 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:17:09.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' 2026-02-19T13:17:09.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:17:09.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-19T13:17:09.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-19T13:17:09.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:17:09.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:17:09.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:17:09.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:17:09.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:17:09.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:17:09.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-snaps/1 2026-02-19T13:17:09.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:17:09.371 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:17:09.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.456002/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:17:09.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-snaps/1/whoami 2026-02-19T13:17:09.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:17:09.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:17:09.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:17:09.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:17:09.391 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:09.390+0000 7fb9a947ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:09.393 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:09.390+0000 7fb9a947ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:09.393 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:09.390+0000 7fb9a947ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:09.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:17:09.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:17:09.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:17:09.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:17:09.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:17:09.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:09.534 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:17:09.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:17:09.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:09.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:17:09.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:17:09.793 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:09.790+0000 7fb9a947ea40 -1 Falling back to public interface 2026-02-19T13:17:10.528 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:10.526+0000 7fb9a947ea40 -1 osd.1 22 log_to_monitors true 2026-02-19T13:17:10.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:17:10.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:10.697 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:17:10.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:17:10.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:10.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:17:10.857 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 29 up_thru 29 down_at 23 last_clean_interval [9,22) [v2:127.0.0.1:6810/1051234064,v1:127.0.0.1:6811/1051234064] [v2:127.0.0.1:6812/1051234064,v1:127.0.0.1:6813/1051234064] exists,up 530c2390-655b-4b76-86a0-719c8728a299 2026-02-19T13:17:10.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:17:10.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:17:10.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:17:10.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:774: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_max 3 2026-02-19T13:17:10.926 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:17:10.926 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_shallow_scrub_chunk_max = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-19T13:17:10.926 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:10.933 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:17:10.933 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_shallow_scrub_chunk_max = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-19T13:17:10.933 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:10.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:775: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_min 3 2026-02-19T13:17:11.014 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:17:11.015 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_min = '' " 2026-02-19T13:17:11.015 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:11.022 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:17:11.022 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_min = '' " 2026-02-19T13:17:11.022 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:11.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:776: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_scrub_chunk_min 3 2026-02-19T13:17:11.104 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:17:11.104 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_chunk_min = '' " 2026-02-19T13:17:11.104 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:11.112 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:17:11.112 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_chunk_min = '' " 2026-02-19T13:17:11.112 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:11.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:777: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_seconds 1 2026-02-19T13:17:11.194 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:17:11.194 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_pg_stat_report_interval_max_seconds = '' " 2026-02-19T13:17:11.194 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:11.201 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:17:11.201 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_pg_stat_report_interval_max_seconds = '' " 2026-02-19T13:17:11.201 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:11.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:778: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_epochs 1 2026-02-19T13:17:11.288 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:17:11.288 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_pg_stat_report_interval_max_epochs = '' " 2026-02-19T13:17:11.288 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:11.295 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:17:11.295 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_pg_stat_report_interval_max_epochs = '' " 2026-02-19T13:17:11.295 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:11.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:779: _scrub_snaps_multi: wait_for_clean 2026-02-19T13:17:11.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:17:11.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:17:11.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:17:11.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:17:11.303 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:17:11.303 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:17:11.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:17:11.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:17:11.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:17:11.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:17:11.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:17:11.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:17:11.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:17:11.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:17:11.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:17:11.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:17:11.586 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:17:11.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:17:11.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:11.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:17:11.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182402 2026-02-19T13:17:11.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182402 2026-02-19T13:17:11.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-107374182402' 2026-02-19T13:17:11.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:11.665 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:17:11.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051586 2026-02-19T13:17:11.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051586 2026-02-19T13:17:11.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-107374182402 1-124554051586' 2026-02-19T13:17:11.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:11.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-107374182402 2026-02-19T13:17:11.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:11.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:17:11.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-107374182402 2026-02-19T13:17:11.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:11.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182402 2026-02-19T13:17:11.747 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 107374182402 2026-02-19T13:17:11.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 107374182402' 2026-02-19T13:17:11.748 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:17:11.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 107374182402 2026-02-19T13:17:11.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:17:12.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:17:12.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:17:13.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182402 -lt 107374182402 2026-02-19T13:17:13.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:13.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-124554051586 2026-02-19T13:17:13.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:13.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:17:13.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-124554051586 2026-02-19T13:17:13.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:13.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051586 2026-02-19T13:17:13.087 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 124554051586 2026-02-19T13:17:13.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 124554051586' 2026-02-19T13:17:13.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:17:13.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051586 -lt 124554051586 2026-02-19T13:17:13.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:17:13.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:17:13.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:17:13.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:17:13.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:17:13.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:17:13.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:17:13.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:17:13.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:17:13.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:17:13.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:17:13.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:17:13.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:17:13.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:17:13.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:17:13.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:17:13.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:17:13.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:17:13.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:781: _scrub_snaps_multi: local pgid=1.0 2026-02-19T13:17:13.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:782: _scrub_snaps_multi: pg_scrub 1.0 2026-02-19T13:17:13.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-19T13:17:13.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-19T13:17:13.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-19T13:17:13.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:17:13.829 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:17:13.829 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:17:13.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:17:13.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:17:13.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:17:14.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:17:14.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:17:14.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:17:14.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:17:14.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:17:14.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:17:14.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:17:14.263 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:17:14.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:17:14.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:14.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:17:14.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182404 2026-02-19T13:17:14.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182404 2026-02-19T13:17:14.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-107374182404' 2026-02-19T13:17:14.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:14.339 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:17:14.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051587 2026-02-19T13:17:14.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051587 2026-02-19T13:17:14.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-107374182404 1-124554051587' 2026-02-19T13:17:14.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:14.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-107374182404 2026-02-19T13:17:14.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:14.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:17:14.417 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-107374182404 2026-02-19T13:17:14.417 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:14.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182404 2026-02-19T13:17:14.419 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 107374182404 2026-02-19T13:17:14.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 107374182404' 2026-02-19T13:17:14.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:17:14.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182402 -lt 107374182404 2026-02-19T13:17:14.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:17:15.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:17:15.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:17:15.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182404 -lt 107374182404 2026-02-19T13:17:15.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:15.762 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-124554051587 2026-02-19T13:17:15.762 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:15.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:17:15.763 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-124554051587 2026-02-19T13:17:15.763 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:15.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051587 2026-02-19T13:17:15.765 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 124554051587 2026-02-19T13:17:15.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 124554051587' 2026-02-19T13:17:15.766 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:17:15.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051587 -lt 124554051587 2026-02-19T13:17:15.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:17:15.938 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 1.0 loop 0 2026-02-19T13:17:15.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-19T13:17:15.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-19T13:17:15.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-19T13:17:15.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:17:15.939 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-19T13:17:15.939 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:17:16.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-19T13:17:16.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:17:16.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:17:16.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:17:16.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 1.0 2026-02-19T13:17:16.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:17:16.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:17:16.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:17:16.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:17:16.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-19T13:16:52.291260+0000 2026-02-19T13:17:16.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 1.0 2026-02-19T13:17:16.335 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.0 on osd.1 to scrub 2026-02-19T13:17:16.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 1.0 2026-02-19T13:16:52.291260+0000 2026-02-19T13:17:16.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T13:17:16.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:16:52.291260+0000 2026-02-19T13:17:16.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:17:16.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:17:16.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:17:16.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:17:16.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:17:16.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:17:16.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:17:16.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:17:16.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:16:52.291260+0000 '>' 2026-02-19T13:16:52.291260+0000 2026-02-19T13:17:16.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:17:17.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:17:17.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:17:17.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:17:17.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:17:17.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:17:17.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:17:17.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:17:17.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:16:52.291260+0000 '>' 2026-02-19T13:16:52.291260+0000 2026-02-19T13:17:17.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:17:18.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:17:18.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:17:18.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:17:18.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:17:18.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:17:18.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:17:18.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:17:18.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:16:52.291260+0000 '>' 2026-02-19T13:16:52.291260+0000 2026-02-19T13:17:18.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:17:19.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:17:19.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:17:19.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:17:19.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:17:19.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:17:19.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:17:19.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:17:20.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:16:52.291260+0000 '>' 2026-02-19T13:16:52.291260+0000 2026-02-19T13:17:20.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:17:21.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:17:21.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-19T13:17:21.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:17:21.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:17:21.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:17:21.003 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:17:21.003 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:17:21.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:17:16.527351+0000 '>' 2026-02-19T13:16:52.291260+0000 2026-02-19T13:17:21.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:17:21.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: grep '_scan_snaps start' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:17:21.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: wc -l 2026-02-19T13:17:21.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: test 16 -gt 3 2026-02-19T13:17:21.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: grep '_scan_snaps start' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:17:21.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: wc -l 2026-02-19T13:17:21.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: test 16 -gt 3 2026-02-19T13:17:21.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:789: _scrub_snaps_multi: rados list-inconsistent-pg test 2026-02-19T13:17:21.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:791: _scrub_snaps_multi: jq '. | length' td/osd-scrub-snaps/json 2026-02-19T13:17:21.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:791: _scrub_snaps_multi: test 1 = 1 2026-02-19T13:17:21.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:793: _scrub_snaps_multi: jq -r '.[0]' td/osd-scrub-snaps/json 2026-02-19T13:17:21.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:793: _scrub_snaps_multi: test 1.0 = 1.0 2026-02-19T13:17:21.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:795: _scrub_snaps_multi: rados list-inconsistent-obj 1.0 --format=json-pretty 2026-02-19T13:17:21.239 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:21.239 INFO:tasks.workunit.client.0.trial144.stdout: "epoch": 29, 2026-02-19T13:17:21.239 INFO:tasks.workunit.client.0.trial144.stdout: "inconsistents": [ 2026-02-19T13:17:21.239 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.239 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:17:21.239 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj1", 2026-02-19T13:17:21.239 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:17:21.239 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:17:21.239 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "version": 18 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [ 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "missing" 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj1", 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 1828249343, 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "version": "16'18", 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "15'1", 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4222.0:1", 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 18, 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:57.074643+0000", 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:57.075056+0000", 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:17:21.240 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x7fc263f2", 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "missing" 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj10", 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:17:21.241 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "version": 32 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj10", 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 718195851, 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "version": "16'32", 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "15'10", 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4243.0:1", 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 32, 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:57.275323+0000", 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:57.275675+0000", 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x7fc263f2", 2026-02-19T13:17:21.242 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "????", 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:17:21.243 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj11", 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "version": 34 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj11", 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:17:21.244 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 693400951, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "version": "16'34", 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "15'11", 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4246.0:1", 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 34, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:57.304163+0000", 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:57.304624+0000", 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x7fc263f2", 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.245 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [] 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj13", 2026-02-19T13:17:21.246 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "version": 38 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj13", 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 2087409765, 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "version": "16'38", 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "15'13", 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4252.0:1", 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 38, 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:57.362168+0000", 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:57.362544+0000", 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:17:21.247 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x7fc263f2", 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]" 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.248 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj14", 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "version": 40 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj14", 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:17:21.249 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 2484217095, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "version": "16'40", 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "15'14", 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4255.0:1", 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 40, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:57.390965+0000", 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:57.391345+0000", 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x7fc263f2", 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:17:21.250 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1033, 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.251 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj15", 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "version": 15 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [ 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_corrupted" 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj15", 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 612772309, 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "version": "15'15", 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "0'0", 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4207.0:1", 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 15, 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:56.403906+0000", 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:56.404193+0000", 2026-02-19T13:17:21.252 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x0678518f", 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_corrupted" 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": "Z2FyYmFnZQo=" 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.253 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 0, 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [] 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj2", 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "version": 56 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [ 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_missing" 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj2", 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 1058988552, 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "version": "22'56", 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "19'48", 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4297.0:1", 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 56, 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "size": 0, 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:58.058814+0000", 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:58.059154+0000", 2026-02-19T13:17:21.254 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "whiteout", 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "dirty" 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0xffffffff", 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_missing" 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "size": 0 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "size": 0, 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.255 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 7, 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 4, 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: 4, 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: 3, 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: 2, 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 7, 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1024, 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: 7, 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: 6, 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: 5 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj3", 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:17:21.256 INFO:tasks.workunit.client.0.trial144.stdout: "version": 44 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "size_mismatch" 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [ 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "size_mismatch_info", 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "obj_size_info_mismatch" 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj3", 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 1643547569, 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "version": "18'44", 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "16'22", 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4267.0:1", 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 44, 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "size": 768, 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:57.600061+0000", 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:57.600406+0000", 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0xad1a2faf", 2026-02-19T13:17:21.257 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "size_mismatch_info", 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "obj_size_info_mismatch" 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "size": 3840, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "object_info": { 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj3", 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 1643547569, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "version": "18'44", 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "16'22", 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4267.0:1", 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 44, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "size": 768, 2026-02-19T13:17:21.258 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:57.600061+0000", 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:57.600406+0000", 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0xad1a2faf", 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "size": 768 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj6", 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:17:21.259 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "version": 24 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj6", 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 2202164420, 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "version": "16'24", 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "15'6", 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4231.0:1", 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 24, 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:57.158765+0000", 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:57.159155+0000", 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x7fc263f2", 2026-02-19T13:17:21.260 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [] 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:17:21.261 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj7", 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "version": 26 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj7", 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 1552453721, 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "version": "16'26", 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "15'7", 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4234.0:1", 2026-02-19T13:17:21.262 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 26, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:57.187391+0000", 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:57.187754+0000", 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x7fc263f2", 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 0, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [] 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.263 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj8", 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "version": 28 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj8", 2026-02-19T13:17:21.264 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 2381834917, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "version": "16'28", 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "15'8", 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4237.0:1", 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 28, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:57.215839+0000", 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:57.216121+0000", 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x7fc263f2", 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.265 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 0, 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.266 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "object": { 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "name": "obj9", 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "nspace": "", 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "locator": "", 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "snap": "head", 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "version": 30 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [ 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "snapset_inconsistency" 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "union_shard_errors": [], 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "selected_object_info": { 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "oid": { 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "oid": "obj9", 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "key": "", 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "snapid": -2, 2026-02-19T13:17:21.267 INFO:tasks.workunit.client.0.trial144.stdout: "hash": 3833113727, 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "max": 0, 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "pool": 1, 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "namespace": "" 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "version": "16'30", 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "prior_version": "15'9", 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "last_reqid": "client.4240.0:1", 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "user_version": 30, 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "mtime": "2026-02-19T13:16:57.246970+0000", 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "local_mtime": "2026-02-19T13:16:57.247309+0000", 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "lost": 0, 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "flags": [ 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "dirty", 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest" 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_seq": 0, 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "truncate_size": 0, 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "data_digest": "0x7fc263f2", 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "omap_digest": "0xffffffff", 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "expected_object_size": 0, 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "expected_write_size": 0, 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "alloc_hint_flags": 0, 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "manifest": { 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "type": 0 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "watchers": {}, 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "shard_versions": [] 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "shards": [ 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.268 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 0, 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "primary": false, 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "size": "????", 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "osd": 1, 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "primary": true, 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "errors": [], 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "size": 256, 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "snapset": { 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "seq": 1, 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "clones": [ 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "snap": 1, 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "size": 1032, 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "overlap": "[]", 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: "snaps": [ 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: 1 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.269 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.270 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.270 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.270 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:21.270 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:17:21.270 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:21.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:797: _scrub_snaps_multi: rados list-inconsistent-snapset 1.0 2026-02-19T13:17:21.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:800: _scrub_snaps_multi: '[' replica = replica ']' 2026-02-19T13:17:21.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:802: _scrub_snaps_multi: scruberrors=20 2026-02-19T13:17:21.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:803: _scrub_snaps_multi: jq .inconsistents 2026-02-19T13:17:21.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:803: _scrub_snaps_multi: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-19T13:17:21.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1066: _scrub_snaps_multi: jq .inconsistents td/osd-scrub-snaps/json 2026-02-19T13:17:21.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1066: _scrub_snaps_multi: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-19T13:17:21.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1067: _scrub_snaps_multi: multidiff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-19T13:17:21.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: multidiff: diff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-19T13:17:21.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1068: _scrub_snaps_multi: test no = yes 2026-02-19T13:17:21.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1073: _scrub_snaps_multi: test '' = yes 2026-02-19T13:17:21.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: find td/osd-scrub-snaps 2026-02-19T13:17:21.296 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: grep 'osd[^/]*\.pid' 2026-02-19T13:17:21.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: pidfiles='td/osd-scrub-snaps/osd.1.pid 2026-02-19T13:17:21.299 INFO:tasks.workunit.client.0.trial144.stderr:td/osd-scrub-snaps/osd.0.pid' 2026-02-19T13:17:21.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1079: _scrub_snaps_multi: pids= 2026-02-19T13:17:21.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1080: _scrub_snaps_multi: for pidfile in ${pidfiles} 2026-02-19T13:17:21.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: cat td/osd-scrub-snaps/osd.1.pid 2026-02-19T13:17:21.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: pids+='509723 ' 2026-02-19T13:17:21.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1080: _scrub_snaps_multi: for pidfile in ${pidfiles} 2026-02-19T13:17:21.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: cat td/osd-scrub-snaps/osd.0.pid 2026-02-19T13:17:21.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: pids+='509181 ' 2026-02-19T13:17:21.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1085: _scrub_snaps_multi: ERRORS=0 2026-02-19T13:17:21.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1089: _scrub_snaps_multi: '[' replica = primary ']' 2026-02-19T13:17:21.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1111: _scrub_snaps_multi: ceph pg dump pgs 2026-02-19T13:17:21.455 INFO:tasks.workunit.client.0.trial144.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T13:17:21.456 INFO:tasks.workunit.client.0.trial144.stdout:1.0 36 0 0 0 0 24448 0 0 56 0 56 active+clean+inconsistent 2026-02-19T13:17:16.527421+0000 22'56 30:103 [1,0] 1 [1,0] 1 22'56 2026-02-19T13:17:16.527351+0000 0'0 2026-02-19T13:16:52.291260+0000 0 1 periodic scrub scheduled @ 2026-02-20T14:35:04.025674+0000 36 0 2026-02-19T13:17:21.456 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T13:17:21.456 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T13:17:21.456 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:17:21.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1113: _scrub_snaps_multi: for pid in $pids 2026-02-19T13:17:21.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1115: _scrub_snaps_multi: kill -0 509723 2026-02-19T13:17:21.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1113: _scrub_snaps_multi: for pid in $pids 2026-02-19T13:17:21.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1115: _scrub_snaps_multi: kill -0 509181 2026-02-19T13:17:21.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1122: _scrub_snaps_multi: kill_daemons td/osd-scrub-snaps 2026-02-19T13:17:21.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:17:21.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:17:21.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:17:21.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:17:21.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:17:26.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:17:26.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1124: _scrub_snaps_multi: declare -a err_strings 2026-02-19T13:17:26.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1125: _scrub_snaps_multi: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj4:7 : missing' 2026-02-19T13:17:26.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1126: _scrub_snaps_multi: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj3:head : size 3840 != size 768 from auth oi' 2026-02-19T13:17:26.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1127: _scrub_snaps_multi: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:1 : missing' 2026-02-19T13:17:26.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1128: _scrub_snaps_multi: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:2 : missing' 2026-02-19T13:17:26.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1129: _scrub_snaps_multi: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj5:4 : size 4608 != size 512 from auth oi' 2026-02-19T13:17:26.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1130: _scrub_snaps_multi: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid .*:::obj5:7 : failed to pick suitable object info' 2026-02-19T13:17:26.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1131: _scrub_snaps_multi: err_strings[6]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj1:head : missing' 2026-02-19T13:17:26.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1132: _scrub_snaps_multi: err_strings[7]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 20 errors' 2026-02-19T13:17:26.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:17:26.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj4:7 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:17:26.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:17:26.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj3:head : size 3840 != size 768 from auth oi' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:17:26.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:17:26.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:1 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:17:26.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:17:26.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:2 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:17:26.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:17:26.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj5:4 : size 4608 != size 512 from auth oi' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:17:26.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:17:26.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid .*:::obj5:7 : failed to pick suitable object info' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:17:26.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:17:26.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj1:head : missing' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:17:26.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-19T13:17:26.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 20 errors' td/osd-scrub-snaps/osd.1.log 2026-02-19T13:17:26.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1144: _scrub_snaps_multi: declare -a rep_err_strings 2026-02-19T13:17:26.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: eval echo '$replica' 2026-02-19T13:17:26.803 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: echo 0 2026-02-19T13:17:26.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: osd=0 2026-02-19T13:17:26.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1146: _scrub_snaps_multi: rep_err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' 2026-02-19T13:17:26.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1147: _scrub_snaps_multi: for err_string in "${rep_err_strings[@]}" 2026-02-19T13:17:26.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1149: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' td/osd-scrub-snaps/osd.0.log 2026-02-19T13:17:26.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1156: _scrub_snaps_multi: '[' 0 '!=' 0 ']' 2026-02-19T13:17:26.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1162: _scrub_snaps_multi: echo 'TEST PASSED' 2026-02-19T13:17:26.807 INFO:tasks.workunit.client.0.trial144.stdout:TEST PASSED 2026-02-19T13:17:26.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1163: _scrub_snaps_multi: return 0 2026-02-19T13:17:26.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1171: TEST_scrub_snaps_replica: err=0 2026-02-19T13:17:26.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1172: TEST_scrub_snaps_replica: CEPH_ARGS='--fsid=a0f216a3-dda2-4e03-883b-1afa9fbf266f --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-19T13:17:26.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1173: TEST_scrub_snaps_replica: return 0 2026-02-19T13:17:26.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:40: run: teardown td/osd-scrub-snaps 2026-02-19T13:17:26.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-snaps 2026-02-19T13:17:26.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:17:26.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-19T13:17:26.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:17:26.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:17:26.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:17:26.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:17:26.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:17:26.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:17:26.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:17:26.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:17:26.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:17:26.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:17:26.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:17:26.817 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:17:26.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:17:26.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:17:26.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:17:26.819 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:17:26.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:17:26.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:17:26.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-snaps 2026-02-19T13:17:26.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:17:26.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:26.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:17:26.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.456002 2026-02-19T13:17:26.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:17:26.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:17:26.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-19T13:17:26.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/osd-scrub-snaps 0 2026-02-19T13:17:26.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-snaps 2026-02-19T13:17:26.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-19T13:17:26.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-19T13:17:26.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:17:26.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:17:26.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:17:26.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:17:26.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:17:26.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:17:26.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:17:26.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:17:26.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:17:26.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:17:26.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:17:26.841 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:17:26.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:17:26.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:17:26.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:17:26.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:17:26.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:17:26.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-19T13:17:26.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-snaps 2026-02-19T13:17:26.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:17:26.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:26.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.456002 2026-02-19T13:17:26.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.456002 2026-02-19T13:17:26.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:17:26.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:17:26.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-19T13:17:26.849 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-19T13:17:26.849 DEBUG:teuthology.orchestra.run.trial144:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-19T13:17:26.899 INFO:tasks.workunit:Running workunit scrub/osd-scrub-test.sh... 2026-02-19T13:17:26.899 DEBUG:teuthology.orchestra.run.trial144:workunit test scrub/osd-scrub-test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=624b9a7bcb2064ce17206482df6a1da007ff7675 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh 2026-02-19T13:17:26.952 INFO:tasks.workunit.client.0.trial144.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-19T13:17:26.958 INFO:tasks.workunit.client.0.trial144.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-19T13:17:26.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-19T13:17:26.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-19T13:17:26.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-19T13:17:26.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-19T13:17:26.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-19T13:17:26.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-19T13:17:26.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-19T13:17:26.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-19T13:17:26.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/osd-scrub-test 2026-02-19T13:17:26.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:21: run: local dir=td/osd-scrub-test 2026-02-19T13:17:26.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:22: run: shift 2026-02-19T13:17:26.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:24: run: export CEPH_MON=127.0.0.1:7138 2026-02-19T13:17:26.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:24: run: CEPH_MON=127.0.0.1:7138 2026-02-19T13:17:26.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:25: run: export CEPH_ARGS 2026-02-19T13:17:26.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:26: run: uuidgen 2026-02-19T13:17:26.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:26: run: CEPH_ARGS+='--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none ' 2026-02-19T13:17:26.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:27: run: CEPH_ARGS+='--mon-host=127.0.0.1:7138 ' 2026-02-19T13:17:26.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:29: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-19T13:17:26.961 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:30: run: set 2026-02-19T13:17:26.961 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-19T13:17:26.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:30: run: local 'funcs=TEST_abort_periodic_for_operator 2026-02-19T13:17:26.962 INFO:tasks.workunit.client.0.trial144.stderr:TEST_deep_scrub_abort 2026-02-19T13:17:26.962 INFO:tasks.workunit.client.0.trial144.stderr:TEST_dump_scrub_schedule 2026-02-19T13:17:26.963 INFO:tasks.workunit.client.0.trial144.stderr:TEST_interval_changes 2026-02-19T13:17:26.963 INFO:tasks.workunit.client.0.trial144.stderr:TEST_just_deep_scrubs 2026-02-19T13:17:26.963 INFO:tasks.workunit.client.0.trial144.stderr:TEST_pg_dump_objects_scrubbed 2026-02-19T13:17:26.963 INFO:tasks.workunit.client.0.trial144.stderr:TEST_scrub_abort 2026-02-19T13:17:26.963 INFO:tasks.workunit.client.0.trial144.stderr:TEST_scrub_permit_time 2026-02-19T13:17:26.963 INFO:tasks.workunit.client.0.trial144.stderr:TEST_scrub_test' 2026-02-19T13:17:26.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-19T13:17:26.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_abort_periodic_for_operator -------------------' 2026-02-19T13:17:26.963 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Prepare Test TEST_abort_periodic_for_operator ------------------- 2026-02-19T13:17:26.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-19T13:17:26.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-test 2026-02-19T13:17:26.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-test 2026-02-19T13:17:26.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:17:26.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:17:26.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:17:26.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:17:26.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:17:26.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:17:26.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:17:26.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:17:26.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:17:26.966 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:17:26.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:17:26.967 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:17:26.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:17:26.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:17:26.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:17:26.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:17:26.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:17:26.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:17:26.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:17:26.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:17:26.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:17:26.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:17:26.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:17:26.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:26.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:17:26.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:17:26.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:17:26.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:17:26.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-test 2026-02-19T13:17:26.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:17:26.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:26.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:17:26.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.511299 2026-02-19T13:17:26.978 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:17:26.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-19T13:17:26.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-19T13:17:26.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:17:26.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-19T13:17:26.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_abort_periodic_for_operator -----------------------' 2026-02-19T13:17:26.980 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Run Test TEST_abort_periodic_for_operator ----------------------- 2026-02-19T13:17:26.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_abort_periodic_for_operator td/osd-scrub-test 2026-02-19T13:17:26.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:748: TEST_abort_periodic_for_operator: local dir=td/osd-scrub-test 2026-02-19T13:17:26.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:749: TEST_abort_periodic_for_operator: cluster_conf=(['osds_num']='5' ['pgs_in_pool']='16' ['pool_name']='test') 2026-02-19T13:17:26.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:749: TEST_abort_periodic_for_operator: local -A cluster_conf 2026-02-19T13:17:26.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:754: TEST_abort_periodic_for_operator: local extr_dbg=1 2026-02-19T13:17:26.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:756: TEST_abort_periodic_for_operator: standard_scrub_wpq_cluster td/osd-scrub-test cluster_conf 2 2026-02-19T13:17:26.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:302: standard_scrub_wpq_cluster: local dir=td/osd-scrub-test 2026-02-19T13:17:26.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:303: standard_scrub_wpq_cluster: local -n conf=cluster_conf 2026-02-19T13:17:26.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:304: standard_scrub_wpq_cluster: local osd_sleep=2 2026-02-19T13:17:26.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:306: standard_scrub_wpq_cluster: conf['extras']=' --osd_op_queue=wpq --osd_scrub_sleep=2 ' 2026-02-19T13:17:26.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:308: standard_scrub_wpq_cluster: standard_scrub_cluster td/osd-scrub-test conf 2026-02-19T13:17:26.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:229: standard_scrub_cluster: local dir=td/osd-scrub-test 2026-02-19T13:17:26.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:230: standard_scrub_cluster: local -n args=conf 2026-02-19T13:17:26.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:232: standard_scrub_cluster: local OSDS=5 2026-02-19T13:17:26.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:233: standard_scrub_cluster: local pg_num=16 2026-02-19T13:17:26.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:235: standard_scrub_cluster: local poolname=test 2026-02-19T13:17:26.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:236: standard_scrub_cluster: args['pool_name']=test 2026-02-19T13:17:26.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:238: standard_scrub_cluster: local pool_default_size=3 2026-02-19T13:17:26.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:239: standard_scrub_cluster: args['pool_default_size']=3 2026-02-19T13:17:26.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:241: standard_scrub_cluster: local 'extra_pars= --osd_op_queue=wpq --osd_scrub_sleep=2 ' 2026-02-19T13:17:26.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:242: standard_scrub_cluster: local debug_msg=dbg 2026-02-19T13:17:26.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:245: standard_scrub_cluster: local saved_echo_flag=x 2026-02-19T13:17:26.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:246: standard_scrub_cluster: set +x 2026-02-19T13:17:27.268 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 a3a7d61b-b19d-474f-bbf6-1f6a4fd682d3 2026-02-19T13:17:27.370 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:17:27.399 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:27.394+0000 7ff410862a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:27.400 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:27.398+0000 7ff410862a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:27.401 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:27.398+0000 7ff410862a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:27.401 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:27.398+0000 7ff410862a40 -1 bdev(0x558723c4cc00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:27.402 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:27.398+0000 7ff410862a40 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-19T13:17:28.123 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:17:28.324 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:17:28.355 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:28.350+0000 7f439d49ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:28.356 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:28.354+0000 7f439d49ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:28.358 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:28.354+0000 7f439d49ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:28.487 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:17:28.781 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:28.778+0000 7f439d49ea40 -1 Falling back to public interface 2026-02-19T13:17:29.405 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:29.402+0000 7f439d49ea40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:17:29.647 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:17:29.802 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4171160161,v1:127.0.0.1:6803/4171160161] [v2:127.0.0.1:6804/4171160161,v1:127.0.0.1:6805/4171160161] exists,up a3a7d61b-b19d-474f-bbf6-1f6a4fd682d3 2026-02-19T13:17:29.807 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 91b57629-2449-4e27-a5e2-ab12a7a46fea 2026-02-19T13:17:29.980 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:17:30.013 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:30.010+0000 7fcea07b8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:30.014 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:30.010+0000 7fcea07b8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:30.016 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:30.014+0000 7fcea07b8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:30.016 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:30.014+0000 7fcea07b8a40 -1 bdev(0x559564bdec00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:30.016 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:30.014+0000 7fcea07b8a40 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-19T13:17:30.643 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:17:30.839 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:17:30.858 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:30.854+0000 7f3212d01a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:30.859 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:30.854+0000 7f3212d01a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:30.860 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:30.858+0000 7f3212d01a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:30.991 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:17:31.189 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:31.186+0000 7f3212d01a40 -1 Falling back to public interface 2026-02-19T13:17:31.820 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:31.818+0000 7f3212d01a40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:17:32.153 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:17:32.311 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/159090769,v1:127.0.0.1:6811/159090769] [v2:127.0.0.1:6812/159090769,v1:127.0.0.1:6813/159090769] exists,up 91b57629-2449-4e27-a5e2-ab12a7a46fea 2026-02-19T13:17:32.317 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 2bba6ba4-cc48-47d9-935d-629c5b834798 2026-02-19T13:17:32.489 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:17:32.529 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:32.526+0000 7f4695822a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:32.531 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:32.526+0000 7f4695822a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:32.532 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:32.530+0000 7f4695822a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:32.532 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:32.530+0000 7f4695822a40 -1 bdev(0x563b24a2ac00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:32.532 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:32.530+0000 7f4695822a40 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-19T13:17:33.170 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T13:17:33.365 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T13:17:33.395 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:33.394+0000 7f8175c7ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:33.397 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:33.394+0000 7f8175c7ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:33.398 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:33.394+0000 7f8175c7ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:33.525 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:17:33.757 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:33.754+0000 7f8175c7ca40 -1 Falling back to public interface 2026-02-19T13:17:34.326 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:34.322+0000 7f8175c7ca40 -1 osd.2 0 log_to_monitors true 2026-02-19T13:17:34.681 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:17:34.835 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3506119394,v1:127.0.0.1:6819/3506119394] [v2:127.0.0.1:6820/3506119394,v1:127.0.0.1:6821/3506119394] exists,up 2bba6ba4-cc48-47d9-935d-629c5b834798 2026-02-19T13:17:34.840 INFO:tasks.workunit.client.0.trial144.stdout:add osd3 19a18c0e-d8a5-485a-a250-005eeafc6d07 2026-02-19T13:17:35.013 INFO:tasks.workunit.client.0.trial144.stdout:3 2026-02-19T13:17:35.044 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:35.042+0000 7f991090ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:35.046 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:35.042+0000 7f991090ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:35.047 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:35.042+0000 7f991090ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:35.048 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:35.046+0000 7f991090ba40 -1 bdev(0x55b3927fec00 td/osd-scrub-test/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:35.048 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:35.046+0000 7f991090ba40 -1 bluestore(td/osd-scrub-test/3) _read_fsid unparsable uuid 2026-02-19T13:17:35.679 INFO:tasks.workunit.client.0.trial144.stdout:adding osd3 key to auth repository 2026-02-19T13:17:35.875 INFO:tasks.workunit.client.0.trial144.stdout:start osd.3 2026-02-19T13:17:35.906 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:35.902+0000 7f6f4301ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:35.908 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:35.906+0000 7f6f4301ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:35.909 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:35.906+0000 7f6f4301ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:36.029 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:17:36.281 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:36.278+0000 7f6f4301ba40 -1 Falling back to public interface 2026-02-19T13:17:36.881 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:36.878+0000 7f6f4301ba40 -1 osd.3 0 log_to_monitors true 2026-02-19T13:17:37.189 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:17:37.347 INFO:tasks.workunit.client.0.trial144.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2515181328,v1:127.0.0.1:6827/2515181328] [v2:127.0.0.1:6828/2515181328,v1:127.0.0.1:6829/2515181328] exists,up 19a18c0e-d8a5-485a-a250-005eeafc6d07 2026-02-19T13:17:37.354 INFO:tasks.workunit.client.0.trial144.stdout:add osd4 53a22350-bbd8-48a8-9c31-34131d4fb0d0 2026-02-19T13:17:37.526 INFO:tasks.workunit.client.0.trial144.stdout:4 2026-02-19T13:17:37.556 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:37.554+0000 7f0900a80a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:37.557 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:37.554+0000 7f0900a80a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:37.558 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:37.554+0000 7f0900a80a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:37.559 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:37.558+0000 7f0900a80a40 -1 bdev(0x5651be2d0c00 td/osd-scrub-test/4/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:37.560 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:37.558+0000 7f0900a80a40 -1 bluestore(td/osd-scrub-test/4) _read_fsid unparsable uuid 2026-02-19T13:17:38.175 INFO:tasks.workunit.client.0.trial144.stdout:adding osd4 key to auth repository 2026-02-19T13:17:38.369 INFO:tasks.workunit.client.0.trial144.stdout:start osd.4 2026-02-19T13:17:38.388 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:38.386+0000 7f22fa8f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:38.390 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:38.386+0000 7f22fa8f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:38.391 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:38.386+0000 7f22fa8f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:38.529 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:17:38.833 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:38.830+0000 7f22fa8f1a40 -1 Falling back to public interface 2026-02-19T13:17:39.445 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:17:39.442+0000 7f22fa8f1a40 -1 osd.4 0 log_to_monitors true 2026-02-19T13:17:39.687 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:17:39.842 INFO:tasks.workunit.client.0.trial144.stdout:osd.4 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3973480395,v1:127.0.0.1:6835/3973480395] [v2:127.0.0.1:6836/3973480395,v1:127.0.0.1:6837/3973480395] exists,up 53a22350-bbd8-48a8-9c31-34131d4fb0d0 2026-02-19T13:17:40.009 INFO:tasks.workunit.client.0.trial144.stderr:pool 'test' created 2026-02-19T13:17:41.654 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836492 2026-02-19T13:17:41.822 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672970 2026-02-19T13:17:41.986 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542151 2026-02-19T13:17:42.151 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.3 seq 77309411333 2026-02-19T13:17:42.322 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.4 seq 94489280515 2026-02-19T13:17:44.177 INFO:tasks.workunit.client.0.trial144.stdout:standard_scrub_cluster: dbg: test pool is test 1 2026-02-19T13:17:44.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:757: TEST_abort_periodic_for_operator: local poolid=1 2026-02-19T13:17:44.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:758: TEST_abort_periodic_for_operator: local poolname=test 2026-02-19T13:17:44.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:759: TEST_abort_periodic_for_operator: echo 'Pool: test : 1' 2026-02-19T13:17:44.178 INFO:tasks.workunit.client.0.trial144.stdout:Pool: test : 1 2026-02-19T13:17:44.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:762: TEST_abort_periodic_for_operator: local saved_echo_flag=x 2026-02-19T13:17:44.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:763: TEST_abort_periodic_for_operator: set +x 2026-02-19T13:17:44.179 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:17:44.179 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:17:44.179 INFO:tasks.workunit.client.0.trial144.stderr:320 bytes copied, 5.8529e-05 s, 5.5 MB/s 2026-02-19T13:17:50.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:776: TEST_abort_periodic_for_operator: declare -A pg_pr 2026-02-19T13:17:50.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:777: TEST_abort_periodic_for_operator: declare -A pg_ac 2026-02-19T13:17:50.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:778: TEST_abort_periodic_for_operator: declare -A pg_po 2026-02-19T13:17:50.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:779: TEST_abort_periodic_for_operator: build_pg_dicts td/osd-scrub-test pg_pr pg_ac pg_po - 2026-02-19T13:17:50.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:318: build_pg_dicts: local dir=td/osd-scrub-test 2026-02-19T13:17:50.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:319: build_pg_dicts: local -n pg_primary_dict=pg_pr 2026-02-19T13:17:50.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:320: build_pg_dicts: local -n pg_acting_dict=pg_ac 2026-02-19T13:17:50.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:321: build_pg_dicts: local -n pg_pool_dict=pg_po 2026-02-19T13:17:50.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:322: build_pg_dicts: local infile=- 2026-02-19T13:17:50.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:324: build_pg_dicts: local extr_dbg=0 2026-02-19T13:17:50.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:327: build_pg_dicts: local saved_echo_flag=x 2026-02-19T13:17:50.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:328: build_pg_dicts: set +x 2026-02-19T13:17:50.391 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs_brief 2026-02-19T13:17:51.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:780: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:781: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:17:51.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:782: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:17:51.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:785: TEST_abort_periodic_for_operator: wait_initial_scrubs pg_pr 2026-02-19T13:17:51.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:691: wait_initial_scrubs: local -n pg_to_prim_dict=pg_pr 2026-02-19T13:17:51.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:692: wait_initial_scrubs: local extr_dbg=1 2026-02-19T13:17:51.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:696: wait_initial_scrubs: ceph tell 'osd.*' config set osd_scrub_min_interval 7200 2026-02-19T13:17:51.141 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:17:51.141 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_min_interval = '' " 2026-02-19T13:17:51.141 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.149 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:17:51.149 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_min_interval = '' " 2026-02-19T13:17:51.149 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.156 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:17:51.156 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_min_interval = '' " 2026-02-19T13:17:51.156 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.164 INFO:tasks.workunit.client.0.trial144.stdout:osd.3: { 2026-02-19T13:17:51.164 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_min_interval = '' " 2026-02-19T13:17:51.164 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.171 INFO:tasks.workunit.client.0.trial144.stdout:osd.4: { 2026-02-19T13:17:51.171 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_min_interval = '' " 2026-02-19T13:17:51.171 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:697: wait_initial_scrubs: ceph tell 'osd.*' config set osd_deep_scrub_interval 14400 2026-02-19T13:17:51.256 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:17:51.256 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_deep_scrub_interval = '' " 2026-02-19T13:17:51.257 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.264 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:17:51.264 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_deep_scrub_interval = '' " 2026-02-19T13:17:51.264 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.271 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:17:51.271 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_deep_scrub_interval = '' " 2026-02-19T13:17:51.272 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.279 INFO:tasks.workunit.client.0.trial144.stdout:osd.3: { 2026-02-19T13:17:51.279 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_deep_scrub_interval = '' " 2026-02-19T13:17:51.279 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.287 INFO:tasks.workunit.client.0.trial144.stdout:osd.4: { 2026-02-19T13:17:51.287 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_deep_scrub_interval = '' " 2026-02-19T13:17:51.287 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:698: wait_initial_scrubs: ceph tell 'osd.*' config set osd_max_scrubs 32 2026-02-19T13:17:51.370 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:17:51.370 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_max_scrubs = '' " 2026-02-19T13:17:51.371 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.377 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:17:51.378 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_max_scrubs = '' " 2026-02-19T13:17:51.378 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.385 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:17:51.385 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_max_scrubs = '' " 2026-02-19T13:17:51.385 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.392 INFO:tasks.workunit.client.0.trial144.stdout:osd.3: { 2026-02-19T13:17:51.392 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_max_scrubs = '' " 2026-02-19T13:17:51.392 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.399 INFO:tasks.workunit.client.0.trial144.stdout:osd.4: { 2026-02-19T13:17:51.399 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_max_scrubs = '' " 2026-02-19T13:17:51.399 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:699: wait_initial_scrubs: ceph tell 'osd.*' config set osd_scrub_sleep 1 2026-02-19T13:17:51.482 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:17:51.483 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:17:51.483 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.490 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:17:51.490 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:17:51.490 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.497 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:17:51.497 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:17:51.498 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.504 INFO:tasks.workunit.client.0.trial144.stdout:osd.3: { 2026-02-19T13:17:51.505 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:17:51.505 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.512 INFO:tasks.workunit.client.0.trial144.stdout:osd.4: { 2026-02-19T13:17:51.512 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:17:51.512 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:700: wait_initial_scrubs: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_max 10 2026-02-19T13:17:51.594 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:17:51.595 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_max = '' " 2026-02-19T13:17:51.595 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.602 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:17:51.602 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_max = '' " 2026-02-19T13:17:51.602 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.609 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:17:51.609 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_max = '' " 2026-02-19T13:17:51.609 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.616 INFO:tasks.workunit.client.0.trial144.stdout:osd.3: { 2026-02-19T13:17:51.616 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_max = '' " 2026-02-19T13:17:51.616 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.623 INFO:tasks.workunit.client.0.trial144.stdout:osd.4: { 2026-02-19T13:17:51.623 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_max = '' " 2026-02-19T13:17:51.624 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:701: wait_initial_scrubs: ceph tell 'osd.*' config set osd_scrub_chunk_max 10 2026-02-19T13:17:51.707 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:17:51.707 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_chunk_max = '' " 2026-02-19T13:17:51.707 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.714 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:17:51.714 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_chunk_max = '' " 2026-02-19T13:17:51.714 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.721 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:17:51.721 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_chunk_max = '' " 2026-02-19T13:17:51.721 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.728 INFO:tasks.workunit.client.0.trial144.stdout:osd.3: { 2026-02-19T13:17:51.728 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_chunk_max = '' " 2026-02-19T13:17:51.728 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.735 INFO:tasks.workunit.client.0.trial144.stdout:osd.4: { 2026-02-19T13:17:51.735 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_chunk_max = '' " 2026-02-19T13:17:51.735 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:51.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:51.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.8' 2026-02-19T13:17:51.744 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.8 2026-02-19T13:17:51.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.8 scrub 2026-02-19T13:17:51.819 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:51.819 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:51.820 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:51.820 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:51.820 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:51.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:51.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.9' 2026-02-19T13:17:51.830 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.9 2026-02-19T13:17:51.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.9 scrub 2026-02-19T13:17:51.894 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:51.894 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:51.895 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:51.895 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:51.895 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:51.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:51.902 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.4 2026-02-19T13:17:51.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.4' 2026-02-19T13:17:51.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.4 scrub 2026-02-19T13:17:51.966 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:51.966 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:51.966 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:51.966 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:51.967 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:51.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:51.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:51.974 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.5 2026-02-19T13:17:51.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.5' 2026-02-19T13:17:51.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.5 scrub 2026-02-19T13:17:52.051 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:52.051 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:52.051 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:52.051 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:52.051 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:52.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:52.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:52.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.6' 2026-02-19T13:17:52.059 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.6 2026-02-19T13:17:52.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.6 scrub 2026-02-19T13:17:52.131 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:52.132 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:52.132 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:52.132 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:52.132 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:52.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:52.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:52.141 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.7 2026-02-19T13:17:52.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.7' 2026-02-19T13:17:52.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.7 scrub 2026-02-19T13:17:52.207 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:52.207 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:52.207 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:52.207 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:52.207 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:52.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:52.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:52.215 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.0 2026-02-19T13:17:52.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.0' 2026-02-19T13:17:52.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.0 scrub 2026-02-19T13:17:52.291 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:52.291 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:52.291 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:52.291 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:52.291 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:52.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:52.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:52.299 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.1 2026-02-19T13:17:52.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.1' 2026-02-19T13:17:52.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.1 scrub 2026-02-19T13:17:52.371 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:52.371 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:52.372 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:52.372 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:52.372 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:52.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:52.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:52.379 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.2 2026-02-19T13:17:52.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.2' 2026-02-19T13:17:52.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.2 scrub 2026-02-19T13:17:52.453 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:52.453 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:52.453 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:52.453 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:52.453 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:52.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:52.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:52.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.3' 2026-02-19T13:17:52.463 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.3 2026-02-19T13:17:52.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.3 scrub 2026-02-19T13:17:52.527 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:52.527 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:52.527 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:52.528 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:52.528 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:52.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:52.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:52.535 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.d 2026-02-19T13:17:52.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.d' 2026-02-19T13:17:52.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.d scrub 2026-02-19T13:17:52.611 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:52.611 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:52.612 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:52.612 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:52.612 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:52.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:52.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:52.619 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.e 2026-02-19T13:17:52.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.e' 2026-02-19T13:17:52.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.e scrub 2026-02-19T13:17:52.692 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:52.692 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:52.692 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:52.692 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:52.692 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:52.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:52.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:52.699 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.f 2026-02-19T13:17:52.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.f' 2026-02-19T13:17:52.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.f scrub 2026-02-19T13:17:52.772 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:52.772 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:52.772 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:52.772 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:52.773 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:52.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:52.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:52.780 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.a 2026-02-19T13:17:52.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.a' 2026-02-19T13:17:52.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.a scrub 2026-02-19T13:17:52.844 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:52.844 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:52.844 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:52.844 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:52.844 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:52.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:52.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:52.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.b' 2026-02-19T13:17:52.852 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.b 2026-02-19T13:17:52.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.b scrub 2026-02-19T13:17:52.927 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:52.927 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:52.927 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:52.927 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:52.927 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:52.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:703: wait_initial_scrubs: for pg in "${!pg_to_prim_dict[@]}" 2026-02-19T13:17:52.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:52.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:704: wait_initial_scrubs: echo 'Scheduling initial scrub for 1.c' 2026-02-19T13:17:52.937 INFO:tasks.workunit.client.0.trial144.stdout:Scheduling initial scrub for 1.c 2026-02-19T13:17:52.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:705: wait_initial_scrubs: ceph tell 1.c scrub 2026-02-19T13:17:53.003 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:17:53.003 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:17:53.003 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:17:53.003 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:17:53.003 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:17:53.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:708: wait_initial_scrubs: sleep 1 2026-02-19T13:17:54.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:709: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:54.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:709: wait_initial_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-19T13:17:54.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:710: wait_initial_scrubs: jq '.pg_stats | map(select(.last_scrub_duration == 0)) | map({pgid: .pgid, last_scrub_duration: .last_scrub_duration})' 2026-02-19T13:17:54.156 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:17:54.165 INFO:tasks.workunit.client.0.trial144.stdout:[ 2026-02-19T13:17:54.165 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:54.165 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.f", 2026-02-19T13:17:54.165 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:54.165 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:54.165 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:54.166 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.e", 2026-02-19T13:17:54.166 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:54.166 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:54.166 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:54.166 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.d", 2026-02-19T13:17:54.166 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:54.166 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:54.166 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:54.167 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.0", 2026-02-19T13:17:54.167 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:54.167 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:54.167 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:54.167 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.1", 2026-02-19T13:17:54.167 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:54.167 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:54.167 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:54.167 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.c", 2026-02-19T13:17:54.167 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:54.167 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:54.167 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:54.167 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.3", 2026-02-19T13:17:54.168 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:54.168 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:54.168 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:54.168 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.4", 2026-02-19T13:17:54.168 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:54.168 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:54.168 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:54.168 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.5", 2026-02-19T13:17:54.168 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:54.168 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:54.168 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:54.168 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.6", 2026-02-19T13:17:54.168 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:54.169 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:54.169 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:54.169 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.7", 2026-02-19T13:17:54.169 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:54.169 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:54.169 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:54.169 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.8", 2026-02-19T13:17:54.169 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:54.169 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:54.169 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:54.169 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.9", 2026-02-19T13:17:54.169 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:54.169 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:54.170 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:54.170 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.a", 2026-02-19T13:17:54.170 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:54.170 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:54.170 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:54.170 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.b", 2026-02-19T13:17:54.170 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:54.170 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:54.170 INFO:tasks.workunit.client.0.trial144.stdout:] 2026-02-19T13:17:54.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:712: wait_initial_scrubs: tout=20 2026-02-19T13:17:54.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:713: wait_initial_scrubs: '[' 20 -gt 0 ']' 2026-02-19T13:17:54.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:714: wait_initial_scrubs: sleep 1 2026-02-19T13:17:55.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:715: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:55.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:715: wait_initial_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-19T13:17:55.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:716: wait_initial_scrubs: jq '.pg_stats | map(select(.last_scrub_duration == 0)) | map({pgid: .pgid, last_scrub_duration: .last_scrub_duration})' 2026-02-19T13:17:55.316 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:17:55.325 INFO:tasks.workunit.client.0.trial144.stdout:[ 2026-02-19T13:17:55.325 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:55.325 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.f", 2026-02-19T13:17:55.325 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:55.325 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:55.325 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:55.325 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.e", 2026-02-19T13:17:55.325 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:55.325 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:55.325 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:55.325 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.d", 2026-02-19T13:17:55.325 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:55.325 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.0", 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.1", 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.c", 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.3", 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.4", 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.5", 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.6", 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.7", 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.8", 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.9", 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:55.326 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:55.327 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:55.327 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.a", 2026-02-19T13:17:55.327 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:55.327 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:55.327 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:55.327 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.b", 2026-02-19T13:17:55.327 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:55.327 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:55.327 INFO:tasks.workunit.client.0.trial144.stdout:] 2026-02-19T13:17:55.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-19T13:17:55.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: jq '.pg_stats | map(select(.last_scrub_duration == 0)) | map({pgid: .pgid, last_scrub_duration: .last_scrub_duration})' 2026-02-19T13:17:55.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: wc -l 2026-02-19T13:17:55.472 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:17:55.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: not_done=62 2026-02-19T13:17:55.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:720: wait_initial_scrubs: '[' 62 -le 1 ']' 2026-02-19T13:17:55.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:723: wait_initial_scrubs: not_done=15 2026-02-19T13:17:55.482 INFO:tasks.workunit.client.0.trial144.stdout:Still waiting for 15 PGs to finish initial scrubs (timeout 20) 2026-02-19T13:17:55.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:724: wait_initial_scrubs: echo 'Still waiting for 15 PGs to finish initial scrubs (timeout 20)' 2026-02-19T13:17:55.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:725: wait_initial_scrubs: tout=19 2026-02-19T13:17:55.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:713: wait_initial_scrubs: '[' 19 -gt 0 ']' 2026-02-19T13:17:55.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:714: wait_initial_scrubs: sleep 1 2026-02-19T13:17:56.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:715: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:56.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:715: wait_initial_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-19T13:17:56.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:716: wait_initial_scrubs: jq '.pg_stats | map(select(.last_scrub_duration == 0)) | map({pgid: .pgid, last_scrub_duration: .last_scrub_duration})' 2026-02-19T13:17:56.632 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:17:56.641 INFO:tasks.workunit.client.0.trial144.stdout:[ 2026-02-19T13:17:56.641 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.f", 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.e", 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.0", 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.c", 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.7", 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.a", 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.b", 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:56.642 INFO:tasks.workunit.client.0.trial144.stdout:] 2026-02-19T13:17:56.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-19T13:17:56.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: jq '.pg_stats | map(select(.last_scrub_duration == 0)) | map({pgid: .pgid, last_scrub_duration: .last_scrub_duration})' 2026-02-19T13:17:56.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: wc -l 2026-02-19T13:17:56.789 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:17:56.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: not_done=30 2026-02-19T13:17:56.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:720: wait_initial_scrubs: '[' 30 -le 1 ']' 2026-02-19T13:17:56.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:723: wait_initial_scrubs: not_done=7 2026-02-19T13:17:56.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:724: wait_initial_scrubs: echo 'Still waiting for 7 PGs to finish initial scrubs (timeout 19)' 2026-02-19T13:17:56.799 INFO:tasks.workunit.client.0.trial144.stdout:Still waiting for 7 PGs to finish initial scrubs (timeout 19) 2026-02-19T13:17:56.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:725: wait_initial_scrubs: tout=18 2026-02-19T13:17:56.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:713: wait_initial_scrubs: '[' 18 -gt 0 ']' 2026-02-19T13:17:56.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:714: wait_initial_scrubs: sleep 1 2026-02-19T13:17:57.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:715: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:57.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:715: wait_initial_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-19T13:17:57.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:716: wait_initial_scrubs: jq '.pg_stats | map(select(.last_scrub_duration == 0)) | map({pgid: .pgid, last_scrub_duration: .last_scrub_duration})' 2026-02-19T13:17:57.946 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:17:57.957 INFO:tasks.workunit.client.0.trial144.stdout:[ 2026-02-19T13:17:57.957 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:57.957 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.e", 2026-02-19T13:17:57.957 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:57.957 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:57.957 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:57.957 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.a", 2026-02-19T13:17:57.957 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:57.957 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:57.957 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:57.958 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.b", 2026-02-19T13:17:57.958 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:57.958 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:57.958 INFO:tasks.workunit.client.0.trial144.stdout:] 2026-02-19T13:17:57.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-19T13:17:57.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: jq '.pg_stats | map(select(.last_scrub_duration == 0)) | map({pgid: .pgid, last_scrub_duration: .last_scrub_duration})' 2026-02-19T13:17:57.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: wc -l 2026-02-19T13:17:58.104 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:17:58.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: not_done=14 2026-02-19T13:17:58.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:720: wait_initial_scrubs: '[' 14 -le 1 ']' 2026-02-19T13:17:58.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:723: wait_initial_scrubs: not_done=3 2026-02-19T13:17:58.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:724: wait_initial_scrubs: echo 'Still waiting for 3 PGs to finish initial scrubs (timeout 18)' 2026-02-19T13:17:58.114 INFO:tasks.workunit.client.0.trial144.stdout:Still waiting for 3 PGs to finish initial scrubs (timeout 18) 2026-02-19T13:17:58.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:725: wait_initial_scrubs: tout=17 2026-02-19T13:17:58.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:713: wait_initial_scrubs: '[' 17 -gt 0 ']' 2026-02-19T13:17:58.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:714: wait_initial_scrubs: sleep 1 2026-02-19T13:17:59.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:715: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:17:59.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:715: wait_initial_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-19T13:17:59.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:716: wait_initial_scrubs: jq '.pg_stats | map(select(.last_scrub_duration == 0)) | map({pgid: .pgid, last_scrub_duration: .last_scrub_duration})' 2026-02-19T13:17:59.263 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:17:59.272 INFO:tasks.workunit.client.0.trial144.stdout:[ 2026-02-19T13:17:59.272 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:59.272 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.e", 2026-02-19T13:17:59.272 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:59.272 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:59.272 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:59.272 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.a", 2026-02-19T13:17:59.272 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:59.273 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:17:59.273 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:17:59.273 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.b", 2026-02-19T13:17:59.273 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:17:59.273 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:17:59.273 INFO:tasks.workunit.client.0.trial144.stdout:] 2026-02-19T13:17:59.274 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-19T13:17:59.274 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: jq '.pg_stats | map(select(.last_scrub_duration == 0)) | map({pgid: .pgid, last_scrub_duration: .last_scrub_duration})' 2026-02-19T13:17:59.274 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: wc -l 2026-02-19T13:17:59.423 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:17:59.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: not_done=14 2026-02-19T13:17:59.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:720: wait_initial_scrubs: '[' 14 -le 1 ']' 2026-02-19T13:17:59.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:723: wait_initial_scrubs: not_done=3 2026-02-19T13:17:59.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:724: wait_initial_scrubs: echo 'Still waiting for 3 PGs to finish initial scrubs (timeout 17)' 2026-02-19T13:17:59.435 INFO:tasks.workunit.client.0.trial144.stdout:Still waiting for 3 PGs to finish initial scrubs (timeout 17) 2026-02-19T13:17:59.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:725: wait_initial_scrubs: tout=16 2026-02-19T13:17:59.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:713: wait_initial_scrubs: '[' 16 -gt 0 ']' 2026-02-19T13:17:59.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:714: wait_initial_scrubs: sleep 1 2026-02-19T13:18:00.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:715: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:18:00.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:715: wait_initial_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-19T13:18:00.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:716: wait_initial_scrubs: jq '.pg_stats | map(select(.last_scrub_duration == 0)) | map({pgid: .pgid, last_scrub_duration: .last_scrub_duration})' 2026-02-19T13:18:00.578 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:18:00.588 INFO:tasks.workunit.client.0.trial144.stdout:[ 2026-02-19T13:18:00.588 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:18:00.588 INFO:tasks.workunit.client.0.trial144.stdout: "pgid": "1.b", 2026-02-19T13:18:00.588 INFO:tasks.workunit.client.0.trial144.stdout: "last_scrub_duration": 0 2026-02-19T13:18:00.588 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:18:00.588 INFO:tasks.workunit.client.0.trial144.stdout:] 2026-02-19T13:18:00.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-19T13:18:00.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: jq '.pg_stats | map(select(.last_scrub_duration == 0)) | map({pgid: .pgid, last_scrub_duration: .last_scrub_duration})' 2026-02-19T13:18:00.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: wc -l 2026-02-19T13:18:00.737 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:18:00.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: not_done=6 2026-02-19T13:18:00.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:720: wait_initial_scrubs: '[' 6 -le 1 ']' 2026-02-19T13:18:00.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:723: wait_initial_scrubs: not_done=1 2026-02-19T13:18:00.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:724: wait_initial_scrubs: echo 'Still waiting for 1 PGs to finish initial scrubs (timeout 16)' 2026-02-19T13:18:00.749 INFO:tasks.workunit.client.0.trial144.stdout:Still waiting for 1 PGs to finish initial scrubs (timeout 16) 2026-02-19T13:18:00.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:725: wait_initial_scrubs: tout=15 2026-02-19T13:18:00.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:713: wait_initial_scrubs: '[' 15 -gt 0 ']' 2026-02-19T13:18:00.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:714: wait_initial_scrubs: sleep 1 2026-02-19T13:18:01.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:715: wait_initial_scrubs: (( extr_dbg >= 1 )) 2026-02-19T13:18:01.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:715: wait_initial_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-19T13:18:01.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:716: wait_initial_scrubs: jq '.pg_stats | map(select(.last_scrub_duration == 0)) | map({pgid: .pgid, last_scrub_duration: .last_scrub_duration})' 2026-02-19T13:18:01.899 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:18:01.908 INFO:tasks.workunit.client.0.trial144.stdout:[] 2026-02-19T13:18:01.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-19T13:18:01.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: jq '.pg_stats | map(select(.last_scrub_duration == 0)) | map({pgid: .pgid, last_scrub_duration: .last_scrub_duration})' 2026-02-19T13:18:01.909 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: wc -l 2026-02-19T13:18:02.056 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:18:02.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:718: wait_initial_scrubs: not_done=1 2026-02-19T13:18:02.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:720: wait_initial_scrubs: '[' 1 -le 1 ']' 2026-02-19T13:18:02.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:721: wait_initial_scrubs: break 2026-02-19T13:18:02.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:727: wait_initial_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-19T13:18:02.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:728: wait_initial_scrubs: jq '.pg_stats | map(select(.last_scrub_duration == 0)) | map({pgid: .pgid, last_scrub_duration: .last_scrub_duration})' 2026-02-19T13:18:02.212 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:18:02.222 INFO:tasks.workunit.client.0.trial144.stdout:[] 2026-02-19T13:18:02.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:729: wait_initial_scrubs: (( tout == 0 )) 2026-02-19T13:18:02.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:730: wait_initial_scrubs: return 0 2026-02-19T13:18:02.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:788: TEST_abort_periodic_for_operator: ceph tell 'osd.*' config set osd_max_scrubs 1 2026-02-19T13:18:02.287 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:18:02.287 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_max_scrubs = '' " 2026-02-19T13:18:02.287 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.294 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:18:02.294 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_max_scrubs = '' " 2026-02-19T13:18:02.295 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.301 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:18:02.302 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_max_scrubs = '' " 2026-02-19T13:18:02.302 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.309 INFO:tasks.workunit.client.0.trial144.stdout:osd.3: { 2026-02-19T13:18:02.309 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_max_scrubs = '' " 2026-02-19T13:18:02.309 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.316 INFO:tasks.workunit.client.0.trial144.stdout:osd.4: { 2026-02-19T13:18:02.316 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_max_scrubs = '' " 2026-02-19T13:18:02.316 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:789: TEST_abort_periodic_for_operator: ceph tell 'osd.*' config set osd_stats_update_period_not_scrubbing 1 2026-02-19T13:18:02.399 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:18:02.399 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_stats_update_period_not_scrubbing = '' " 2026-02-19T13:18:02.399 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.406 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:18:02.406 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_stats_update_period_not_scrubbing = '' " 2026-02-19T13:18:02.406 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_stats_update_period_not_scrubbing = '' " 2026-02-19T13:18:02.414 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.421 INFO:tasks.workunit.client.0.trial144.stdout:osd.3: { 2026-02-19T13:18:02.421 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_stats_update_period_not_scrubbing = '' " 2026-02-19T13:18:02.421 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.428 INFO:tasks.workunit.client.0.trial144.stdout:osd.4: { 2026-02-19T13:18:02.428 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_stats_update_period_not_scrubbing = '' " 2026-02-19T13:18:02.428 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:792: TEST_abort_periodic_for_operator: ceph tell 'osd.*' config set osd_scrub_sleep 3 2026-02-19T13:18:02.510 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:18:02.510 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:18:02.511 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.517 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:18:02.518 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:18:02.518 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.524 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:18:02.525 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:18:02.525 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.532 INFO:tasks.workunit.client.0.trial144.stdout:osd.3: { 2026-02-19T13:18:02.532 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:18:02.532 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.539 INFO:tasks.workunit.client.0.trial144.stdout:osd.4: { 2026-02-19T13:18:02.539 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:18:02.539 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:793: TEST_abort_periodic_for_operator: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_max 2 2026-02-19T13:18:02.622 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:18:02.622 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_max = '' " 2026-02-19T13:18:02.623 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.630 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:18:02.630 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_max = '' " 2026-02-19T13:18:02.630 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.637 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:18:02.637 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_max = '' " 2026-02-19T13:18:02.638 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.645 INFO:tasks.workunit.client.0.trial144.stdout:osd.3: { 2026-02-19T13:18:02.645 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_max = '' " 2026-02-19T13:18:02.646 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.652 INFO:tasks.workunit.client.0.trial144.stdout:osd.4: { 2026-02-19T13:18:02.653 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_max = '' " 2026-02-19T13:18:02.653 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:794: TEST_abort_periodic_for_operator: ceph tell 'osd.*' config set osd_scrub_chunk_max 2 2026-02-19T13:18:02.734 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:18:02.735 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_chunk_max = '' " 2026-02-19T13:18:02.735 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.742 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:18:02.742 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_chunk_max = '' " 2026-02-19T13:18:02.742 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.749 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:18:02.749 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_chunk_max = '' " 2026-02-19T13:18:02.749 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.756 INFO:tasks.workunit.client.0.trial144.stdout:osd.3: { 2026-02-19T13:18:02.756 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_chunk_max = '' " 2026-02-19T13:18:02.756 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.763 INFO:tasks.workunit.client.0.trial144.stdout:osd.4: { 2026-02-19T13:18:02.763 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_chunk_max = '' " 2026-02-19T13:18:02.763 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:795: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:18:02.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:798: TEST_abort_periodic_for_operator: local pg2= 2026-02-19T13:18:02.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:799: TEST_abort_periodic_for_operator: for pg1 in "${!pg_pr[@]}" 2026-02-19T13:18:02.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:800: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:18:02.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:801: TEST_abort_periodic_for_operator: [[ 1.8 == \1\.\8 ]] 2026-02-19T13:18:02.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:802: TEST_abort_periodic_for_operator: continue 2026-02-19T13:18:02.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:800: TEST_abort_periodic_for_operator: for pg in "${!pg_pr[@]}" 2026-02-19T13:18:02.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:801: TEST_abort_periodic_for_operator: [[ 1.9 == \1\.\8 ]] 2026-02-19T13:18:02.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:804: TEST_abort_periodic_for_operator: [[ 1 == \1 ]] 2026-02-19T13:18:02.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:805: TEST_abort_periodic_for_operator: count_common_active 1.9 1.8 pg_ac 2026-02-19T13:18:02.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:367: count_common_active: local pg1=1.9 2026-02-19T13:18:02.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:368: count_common_active: local pg2=1.8 2026-02-19T13:18:02.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:369: count_common_active: local -n pg_acting_dict=pg_ac 2026-02-19T13:18:02.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:371: count_common_active: a1=('1' '4' '2') 2026-02-19T13:18:02.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:371: count_common_active: local -a a1 2026-02-19T13:18:02.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:372: count_common_active: a2=('1' '4' '3') 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:372: count_common_active: local -a a2 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:374: count_common_active: local -i cnt=0 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:375: count_common_active: for i in "${a1[@]}" 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:376: count_common_active: for j in "${a2[@]}" 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:377: count_common_active: [[ 1 -eq 1 ]] 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:378: count_common_active: cnt=1 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:376: count_common_active: for j in "${a2[@]}" 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:377: count_common_active: [[ 1 -eq 4 ]] 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:376: count_common_active: for j in "${a2[@]}" 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:377: count_common_active: [[ 1 -eq 3 ]] 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:375: count_common_active: for i in "${a1[@]}" 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:376: count_common_active: for j in "${a2[@]}" 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:377: count_common_active: [[ 4 -eq 1 ]] 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:376: count_common_active: for j in "${a2[@]}" 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:377: count_common_active: [[ 4 -eq 4 ]] 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:378: count_common_active: cnt=2 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:376: count_common_active: for j in "${a2[@]}" 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:377: count_common_active: [[ 4 -eq 3 ]] 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:375: count_common_active: for i in "${a1[@]}" 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:376: count_common_active: for j in "${a2[@]}" 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:377: count_common_active: [[ 2 -eq 1 ]] 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:376: count_common_active: for j in "${a2[@]}" 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:377: count_common_active: [[ 2 -eq 4 ]] 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:376: count_common_active: for j in "${a2[@]}" 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:377: count_common_active: [[ 2 -eq 3 ]] 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:383: count_common_active: printf %d 2 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:805: TEST_abort_periodic_for_operator: local -i common=2 2026-02-19T13:18:02.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:806: TEST_abort_periodic_for_operator: [[ 2 -gt 1 ]] 2026-02-19T13:18:02.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:807: TEST_abort_periodic_for_operator: pg2=1.9 2026-02-19T13:18:02.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:808: TEST_abort_periodic_for_operator: break 2 2026-02-19T13:18:02.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:814: TEST_abort_periodic_for_operator: [[ -z 1.9 ]] 2026-02-19T13:18:02.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:820: TEST_abort_periodic_for_operator: ceph tell osd.1 config set osd_max_scrubs 2 2026-02-19T13:18:02.837 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:18:02.837 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_max_scrubs = '' " 2026-02-19T13:18:02.837 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:02.844 INFO:tasks.workunit.client.0.trial144.stdout:The two PGs to manipulate are 1.8 and 1.9 2026-02-19T13:18:02.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:821: TEST_abort_periodic_for_operator: echo 'The two PGs to manipulate are 1.8 and 1.9' 2026-02-19T13:18:02.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:823: TEST_abort_periodic_for_operator: set_query_debug 1.8 2026-02-19T13:18:02.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:417: set_query_debug: local pgid=1.8 2026-02-19T13:18:02.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:419: set_query_debug: ceph pg dump pgs_brief 2026-02-19T13:18:02.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:419: set_query_debug: awk -v 'pg=^1.8' -n -e '$0 ~ pg { print(gensub(/[^0-9]*([0-9]+).*/,"\\1","g",$5)); }' 2026-02-19T13:18:02.995 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs_brief 2026-02-19T13:18:03.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:419: set_query_debug: local prim_osd=1 2026-02-19T13:18:03.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:421: set_query_debug: echo 'Setting scrub debug data. Primary for 1.8 is 1' 2026-02-19T13:18:03.005 INFO:tasks.workunit.client.0.trial144.stdout:Setting scrub debug data. Primary for 1.8 is 1 2026-02-19T13:18:03.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:422: set_query_debug: get_asok_path osd.1 2026-02-19T13:18:03.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:18:03.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:18:03.007 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:18:03.007 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:03.007 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:18:03.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-osd.1.asok 2026-02-19T13:18:03.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:422: set_query_debug: CEPH_ARGS= 2026-02-19T13:18:03.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:422: set_query_debug: ceph --format=json daemon /tmp/ceph-asok.511299/ceph-osd.1.asok scrubdebug 1.8 set sessions 2026-02-19T13:18:03.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:826: TEST_abort_periodic_for_operator: local is_act 2026-02-19T13:18:03.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:827: TEST_abort_periodic_for_operator: seq 1 3 2026-02-19T13:18:03.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:827: TEST_abort_periodic_for_operator: for i in $( seq 1 3 ) 2026-02-19T13:18:03.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:829: TEST_abort_periodic_for_operator: ceph pg 1.8 query 2026-02-19T13:18:03.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:829: TEST_abort_periodic_for_operator: jq .scrubber.active 2026-02-19T13:18:03.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:829: TEST_abort_periodic_for_operator: is_act=false 2026-02-19T13:18:03.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:830: TEST_abort_periodic_for_operator: [[ false = \f\a\l\s\e ]] 2026-02-19T13:18:03.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:831: TEST_abort_periodic_for_operator: break 2026-02-19T13:18:03.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:836: TEST_abort_periodic_for_operator: ceph pg dump pgs 2026-02-19T13:18:03.293 INFO:tasks.workunit.client.0.trial144.stdout:{"success":true}PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T13:18:03.293 INFO:tasks.workunit.client.0.trial144.stdout:1.f 19 0 0 0 0 6080 0 0 19 0 19 active+clean 2026-02-19T13:17:55.766158+0000 26'19 26:45 [3,0,4] 3 [3,0,4] 3 26'19 2026-02-19T13:17:55.766115+0000 0'0 2026-02-19T13:17:40.007646+0000 0 2 periodic scrub scheduled @ 2026-02-19T15:17:55.766115+0000 19 0 2026-02-19T13:18:03.293 INFO:tasks.workunit.client.0.trial144.stdout:1.e 20 0 0 0 0 6400 0 0 20 0 20 active+clean 2026-02-19T13:17:57.701939+0000 26'20 26:47 [1,4,2] 1 [1,4,2] 1 26'20 2026-02-19T13:17:57.701881+0000 0'0 2026-02-19T13:17:40.007646+0000 0 2 periodic scrub scheduled @ 2026-02-19T15:17:57.701881+0000 20 0 2026-02-19T13:18:03.293 INFO:tasks.workunit.client.0.trial144.stdout:1.d 14 0 0 0 0 4480 0 0 14 0 14 active+clean 2026-02-19T13:17:54.233924+0000 26'14 26:41 [2,4,1] 2 [2,4,1] 2 26'14 2026-02-19T13:17:54.233865+0000 0'0 2026-02-19T13:17:40.007646+0000 0 1 periodic scrub scheduled @ 2026-02-19T15:17:54.233865+0000 14 0 2026-02-19T13:18:03.293 INFO:tasks.workunit.client.0.trial144.stdout:1.0 22 0 0 0 0 7040 0 0 22 0 22 active+clean 2026-02-19T13:17:56.714213+0000 26'22 26:53 [1,0,2] 1 [1,0,2] 1 26'22 2026-02-19T13:17:56.714162+0000 0'0 2026-02-19T13:17:40.007646+0000 0 2 periodic scrub scheduled @ 2026-02-19T15:17:56.714162+0000 22 0 2026-02-19T13:18:03.293 INFO:tasks.workunit.client.0.trial144.stdout:1.1 12 0 0 0 0 3840 0 0 12 0 12 active+clean 2026-02-19T13:17:54.633533+0000 26'12 26:41 [4,3,0] 4 [4,3,0] 4 26'12 2026-02-19T13:17:54.633488+0000 0'0 2026-02-19T13:17:40.007646+0000 0 2 periodic scrub scheduled @ 2026-02-19T15:17:54.633488+0000 12 0 2026-02-19T13:18:03.293 INFO:tasks.workunit.client.0.trial144.stdout:1.c 15 0 0 0 0 4800 0 0 15 0 15 active+clean 2026-02-19T13:17:56.755830+0000 26'15 26:41 [3,0,4] 3 [3,0,4] 3 26'15 2026-02-19T13:17:56.755776+0000 0'0 2026-02-19T13:17:40.007646+0000 0 1 periodic scrub scheduled @ 2026-02-19T15:17:56.755776+0000 15 0 2026-02-19T13:18:03.293 INFO:tasks.workunit.client.0.trial144.stdout:1.2 15 0 0 0 0 4800 0 0 15 0 15 active+clean 2026-02-19T13:17:53.487628+0000 26'15 26:43 [0,4,1] 0 [0,4,1] 0 26'15 2026-02-19T13:17:53.487534+0000 0'0 2026-02-19T13:17:40.007646+0000 0 1 periodic scrub scheduled @ 2026-02-19T15:17:53.487534+0000 15 0 2026-02-19T13:18:03.293 INFO:tasks.workunit.client.0.trial144.stdout:1.3 19 0 0 0 0 6080 0 0 19 0 19 active+clean 2026-02-19T13:17:55.595308+0000 26'19 26:47 [4,1,2] 4 [4,1,2] 4 26'19 2026-02-19T13:17:55.595247+0000 0'0 2026-02-19T13:17:40.007646+0000 0 1 periodic scrub scheduled @ 2026-02-19T15:17:55.595247+0000 19 0 2026-02-19T13:18:03.293 INFO:tasks.workunit.client.0.trial144.stdout:1.4 15 0 0 0 0 4800 0 0 15 0 15 active+clean 2026-02-19T13:17:53.759607+0000 26'15 26:43 [3,0,1] 3 [3,0,1] 3 26'15 2026-02-19T13:17:53.759547+0000 0'0 2026-02-19T13:17:40.007646+0000 0 1 periodic scrub scheduled @ 2026-02-19T15:17:53.759547+0000 15 0 2026-02-19T13:18:03.294 INFO:tasks.workunit.client.0.trial144.stdout:1.5 26 0 0 0 0 8320 0 0 26 0 26 active+clean 2026-02-19T13:17:53.611575+0000 26'26 26:53 [4,2,3] 4 [4,2,3] 4 26'26 2026-02-19T13:17:53.611519+0000 0'0 2026-02-19T13:17:40.007646+0000 0 1 periodic scrub scheduled @ 2026-02-19T15:17:53.611519+0000 26 0 2026-02-19T13:18:03.294 INFO:tasks.workunit.client.0.trial144.stdout:1.6 12 0 0 0 0 3840 0 0 12 0 12 active+clean 2026-02-19T13:17:54.783810+0000 26'12 26:39 [3,4,2] 3 [3,4,2] 3 26'12 2026-02-19T13:17:54.783768+0000 0'0 2026-02-19T13:17:40.007646+0000 0 1 periodic scrub scheduled @ 2026-02-19T15:17:54.783768+0000 12 0 2026-02-19T13:18:03.294 INFO:tasks.workunit.client.0.trial144.stdout:1.7 11 0 0 0 0 3520 0 0 11 0 11 active+clean 2026-02-19T13:17:55.702163+0000 26'11 26:39 [1,3,0] 1 [1,3,0] 1 26'11 2026-02-19T13:17:55.702078+0000 0'0 2026-02-19T13:17:40.007646+0000 0 2 periodic scrub scheduled @ 2026-02-19T15:17:55.702078+0000 11 0 2026-02-19T13:18:03.294 INFO:tasks.workunit.client.0.trial144.stdout:1.8 28 0 0 0 0 8960 0 0 28 0 28 active+clean 2026-02-19T13:17:53.667982+0000 26'28 26:55 [1,4,3] 1 [1,4,3] 1 26'28 2026-02-19T13:17:53.667931+0000 0'0 2026-02-19T13:17:40.007646+0000 0 1 periodic scrub scheduled @ 2026-02-19T15:17:53.667931+0000 28 0 2026-02-19T13:18:03.294 INFO:tasks.workunit.client.0.trial144.stdout:1.9 8 0 0 0 0 2560 0 0 8 0 8 active+clean 2026-02-19T13:17:54.693255+0000 26'8 26:35 [1,4,2] 1 [1,4,2] 1 26'8 2026-02-19T13:17:54.693191+0000 0'0 2026-02-19T13:17:40.007646+0000 0 2 periodic scrub scheduled @ 2026-02-19T15:17:54.693191+0000 8 0 2026-02-19T13:18:03.294 INFO:tasks.workunit.client.0.trial144.stdout:1.a 9 0 0 0 0 2880 0 0 9 0 9 active+clean 2026-02-19T13:17:58.679038+0000 26'9 26:37 [1,2,4] 1 [1,2,4] 1 26'9 2026-02-19T13:17:58.678971+0000 0'0 2026-02-19T13:17:40.007646+0000 0 1 periodic scrub scheduled @ 2026-02-19T15:17:58.678971+0000 9 0 2026-02-19T13:18:03.294 INFO:tasks.workunit.client.0.trial144.stdout:1.b 11 0 0 0 0 3520 0 0 11 0 11 active+clean 2026-02-19T13:17:59.651159+0000 26'11 26:39 [1,0,2] 1 [1,0,2] 1 26'11 2026-02-19T13:17:59.651065+0000 0'0 2026-02-19T13:17:40.007646+0000 0 1 periodic scrub scheduled @ 2026-02-19T15:17:59.651065+0000 11 0 2026-02-19T13:18:03.294 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T13:18:03.294 INFO:tasks.workunit.client.0.trial144.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T13:18:03.294 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:18:03.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:837: TEST_abort_periodic_for_operator: [[ false != \f\a\l\s\e ]] 2026-02-19T13:18:03.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:842: TEST_abort_periodic_for_operator: sleep 0.5 2026-02-19T13:18:03.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:844: TEST_abort_periodic_for_operator: echo 'Initiating a periodic scrub of 1.8' 2026-02-19T13:18:03.806 INFO:tasks.workunit.client.0.trial144.stdout:Initiating a periodic scrub of 1.8 2026-02-19T13:18:03.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:845: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:18:03.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:846: TEST_abort_periodic_for_operator: ceph tell 1.8 schedule-deep-scrub 2026-02-19T13:18:03.871 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:18:03.871 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T13:18:03.872 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:18:03.872 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-19T06:53:53.872096+0000" 2026-02-19T13:18:03.872 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:03.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:847: TEST_abort_periodic_for_operator: sleep 1 2026-02-19T13:18:04.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:848: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:18:04.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:850: TEST_abort_periodic_for_operator: seq 1 14 2026-02-19T13:18:04.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:850: TEST_abort_periodic_for_operator: for i in $( seq 1 14 ) 2026-02-19T13:18:04.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:852: TEST_abort_periodic_for_operator: sleep 0.5 2026-02-19T13:18:05.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:853: TEST_abort_periodic_for_operator: ceph pg 1.8 query 2026-02-19T13:18:05.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:853: TEST_abort_periodic_for_operator: jq .scrubber 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:853: TEST_abort_periodic_for_operator: stt='{ 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "active": true, 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "epoch_start": "23", 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "start": "1:10000000::::0", 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "end": "MIN", 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "max_end": "MIN", 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "subset_last_update": "0'\''0", 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "deep": true, 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "req_scrub": false, 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "auto_repair": false, 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "check_repair": false, 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "deep_scrub_on_error": false, 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "priority": 5, 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "shallow_errors": 0, 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "deep_errors": 0, 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "fixed": 0, 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "waiting_on_whom": [], 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "schedule": "scrubbing", 2026-02-19T13:18:05.459 INFO:tasks.workunit.client.0.trial144.stderr: "is_reserving_replicas": false, 2026-02-19T13:18:05.460 INFO:tasks.workunit.client.0.trial144.stderr: "test_sequence": 10850308 2026-02-19T13:18:05.460 INFO:tasks.workunit.client.0.trial144.stderr:}' 2026-02-19T13:18:05.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:854: TEST_abort_periodic_for_operator: jq .active 2026-02-19T13:18:05.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:854: TEST_abort_periodic_for_operator: echo '{' '"active":' true, '"epoch_start":' '"23",' '"start":' '"1:10000000::::0",' '"end":' '"MIN",' '"max_end":' '"MIN",' '"subset_last_update":' '"0'\''0",' '"deep":' true, '"req_scrub":' false, '"auto_repair":' false, '"check_repair":' false, '"deep_scrub_on_error":' false, '"priority":' 5, '"shallow_errors":' 0, '"deep_errors":' 0, '"fixed":' 0, '"waiting_on_whom":' '[],' '"schedule":' '"scrubbing",' '"is_reserving_replicas":' false, '"test_sequence":' 10850308 '}' 2026-02-19T13:18:05.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:854: TEST_abort_periodic_for_operator: is_active=true 2026-02-19T13:18:05.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:855: TEST_abort_periodic_for_operator: echo '{' '"active":' true, '"epoch_start":' '"23",' '"start":' '"1:10000000::::0",' '"end":' '"MIN",' '"max_end":' '"MIN",' '"subset_last_update":' '"0'\''0",' '"deep":' true, '"req_scrub":' false, '"auto_repair":' false, '"check_repair":' false, '"deep_scrub_on_error":' false, '"priority":' 5, '"shallow_errors":' 0, '"deep_errors":' 0, '"fixed":' 0, '"waiting_on_whom":' '[],' '"schedule":' '"scrubbing",' '"is_reserving_replicas":' false, '"test_sequence":' 10850308 '}' 2026-02-19T13:18:05.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:855: TEST_abort_periodic_for_operator: jq .is_reserving_replicas 2026-02-19T13:18:05.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:855: TEST_abort_periodic_for_operator: is_reserving_replicas=false 2026-02-19T13:18:05.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:856: TEST_abort_periodic_for_operator: [[ true = \t\r\u\e ]] 2026-02-19T13:18:05.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:856: TEST_abort_periodic_for_operator: [[ false = \f\a\l\s\e ]] 2026-02-19T13:18:05.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:857: TEST_abort_periodic_for_operator: break 2026-02-19T13:18:05.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:861: TEST_abort_periodic_for_operator: [[ true != \t\r\u\e ]] 2026-02-19T13:18:05.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:861: TEST_abort_periodic_for_operator: [[ false != \f\a\l\s\e ]] 2026-02-19T13:18:05.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:866: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:18:05.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:873: TEST_abort_periodic_for_operator: (( extr_dbg >= 1 )) 2026-02-19T13:18:05.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:873: TEST_abort_periodic_for_operator: ceph tell 'osd.*' dump_scrub_reservations --format=json-pretty 2026-02-19T13:18:05.541 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T13:18:05.541 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:18:05.542 INFO:tasks.workunit.client.0.trial144.stdout: "scrubs_local": 0, 2026-02-19T13:18:05.542 INFO:tasks.workunit.client.0.trial144.stdout: "osd_max_scrubs": 1, 2026-02-19T13:18:05.542 INFO:tasks.workunit.client.0.trial144.stdout: "remote_scrub_reservations": { 2026-02-19T13:18:05.542 INFO:tasks.workunit.client.0.trial144.stdout: "max_allowed": 1, 2026-02-19T13:18:05.542 INFO:tasks.workunit.client.0.trial144.stdout: "min_priority": 0, 2026-02-19T13:18:05.542 INFO:tasks.workunit.client.0.trial144.stdout: "queues": [], 2026-02-19T13:18:05.542 INFO:tasks.workunit.client.0.trial144.stdout: "in_progress": [] 2026-02-19T13:18:05.542 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:18:05.542 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:05.549 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T13:18:05.549 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:18:05.549 INFO:tasks.workunit.client.0.trial144.stdout: "scrubs_local": 1, 2026-02-19T13:18:05.549 INFO:tasks.workunit.client.0.trial144.stdout: "osd_max_scrubs": 2, 2026-02-19T13:18:05.549 INFO:tasks.workunit.client.0.trial144.stdout: "remote_scrub_reservations": { 2026-02-19T13:18:05.550 INFO:tasks.workunit.client.0.trial144.stdout: "max_allowed": 2, 2026-02-19T13:18:05.550 INFO:tasks.workunit.client.0.trial144.stdout: "min_priority": 0, 2026-02-19T13:18:05.550 INFO:tasks.workunit.client.0.trial144.stdout: "queues": [], 2026-02-19T13:18:05.550 INFO:tasks.workunit.client.0.trial144.stdout: "in_progress": [] 2026-02-19T13:18:05.550 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:18:05.550 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:05.557 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T13:18:05.557 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:18:05.557 INFO:tasks.workunit.client.0.trial144.stdout: "scrubs_local": 0, 2026-02-19T13:18:05.557 INFO:tasks.workunit.client.0.trial144.stdout: "osd_max_scrubs": 1, 2026-02-19T13:18:05.557 INFO:tasks.workunit.client.0.trial144.stdout: "remote_scrub_reservations": { 2026-02-19T13:18:05.557 INFO:tasks.workunit.client.0.trial144.stdout: "max_allowed": 1, 2026-02-19T13:18:05.557 INFO:tasks.workunit.client.0.trial144.stdout: "min_priority": 0, 2026-02-19T13:18:05.557 INFO:tasks.workunit.client.0.trial144.stdout: "queues": [], 2026-02-19T13:18:05.558 INFO:tasks.workunit.client.0.trial144.stdout: "in_progress": [] 2026-02-19T13:18:05.558 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:18:05.558 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:05.565 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T13:18:05.565 INFO:tasks.workunit.client.0.trial144.stdout:osd.3: { 2026-02-19T13:18:05.565 INFO:tasks.workunit.client.0.trial144.stdout: "scrubs_local": 0, 2026-02-19T13:18:05.565 INFO:tasks.workunit.client.0.trial144.stdout: "osd_max_scrubs": 1, 2026-02-19T13:18:05.565 INFO:tasks.workunit.client.0.trial144.stdout: "remote_scrub_reservations": { 2026-02-19T13:18:05.565 INFO:tasks.workunit.client.0.trial144.stdout: "max_allowed": 1, 2026-02-19T13:18:05.565 INFO:tasks.workunit.client.0.trial144.stdout: "min_priority": 0, 2026-02-19T13:18:05.565 INFO:tasks.workunit.client.0.trial144.stdout: "queues": [], 2026-02-19T13:18:05.565 INFO:tasks.workunit.client.0.trial144.stdout: "in_progress": [ 2026-02-19T13:18:05.565 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:18:05.565 INFO:tasks.workunit.client.0.trial144.stdout: "item": "1.8", 2026-02-19T13:18:05.565 INFO:tasks.workunit.client.0.trial144.stdout: "prio": 0, 2026-02-19T13:18:05.565 INFO:tasks.workunit.client.0.trial144.stdout: "can_preempt": false 2026-02-19T13:18:05.565 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:18:05.566 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:18:05.566 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:18:05.566 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout: 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout:osd.4: { 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout: "scrubs_local": 0, 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout: "osd_max_scrubs": 1, 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout: "remote_scrub_reservations": { 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout: "max_allowed": 1, 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout: "min_priority": 0, 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout: "queues": [], 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout: "in_progress": [ 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout: "item": "1.8", 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout: "prio": 0, 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout: "can_preempt": false 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:18:05.573 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:05.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:876: TEST_abort_periodic_for_operator: set_query_debug 1.9 2026-02-19T13:18:05.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:417: set_query_debug: local pgid=1.9 2026-02-19T13:18:05.583 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:419: set_query_debug: ceph pg dump pgs_brief 2026-02-19T13:18:05.583 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:419: set_query_debug: awk -v 'pg=^1.9' -n -e '$0 ~ pg { print(gensub(/[^0-9]*([0-9]+).*/,"\\1","g",$5)); }' 2026-02-19T13:18:05.728 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs_brief 2026-02-19T13:18:05.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:419: set_query_debug: local prim_osd=1 2026-02-19T13:18:05.739 INFO:tasks.workunit.client.0.trial144.stdout:Setting scrub debug data. Primary for 1.9 is 1 2026-02-19T13:18:05.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:421: set_query_debug: echo 'Setting scrub debug data. Primary for 1.9 is 1' 2026-02-19T13:18:05.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:422: set_query_debug: get_asok_path osd.1 2026-02-19T13:18:05.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:18:05.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:18:05.740 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:18:05.740 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:05.740 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:18:05.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-osd.1.asok 2026-02-19T13:18:05.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:422: set_query_debug: CEPH_ARGS= 2026-02-19T13:18:05.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:422: set_query_debug: ceph --format=json daemon /tmp/ceph-asok.511299/ceph-osd.1.asok scrubdebug 1.9 set sessions 2026-02-19T13:18:05.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:877: TEST_abort_periodic_for_operator: ceph tell 1.9 schedule-deep-scrub 2026-02-19T13:18:05.887 INFO:tasks.workunit.client.0.trial144.stdout:{"success":true}{ 2026-02-19T13:18:05.887 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T13:18:05.888 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:18:05.888 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-19T06:53:55.888500+0000" 2026-02-19T13:18:05.888 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:05.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:878: TEST_abort_periodic_for_operator: sleep 0.5 2026-02-19T13:18:06.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:879: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:18:06.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:880: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:18:06.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:881: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:18:06.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:882: TEST_abort_periodic_for_operator: sleep 1 2026-02-19T13:18:07.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:883: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:18:07.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:884: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:18:07.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:885: TEST_abort_periodic_for_operator: (( extr_dbg >= 2 )) 2026-02-19T13:18:07.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:888: TEST_abort_periodic_for_operator: ceph pg 1.9 query 2026-02-19T13:18:07.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:888: TEST_abort_periodic_for_operator: jq .scrubber 2026-02-19T13:18:07.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:888: TEST_abort_periodic_for_operator: local 'stt2={ 2026-02-19T13:18:07.475 INFO:tasks.workunit.client.0.trial144.stderr: "active": true, 2026-02-19T13:18:07.475 INFO:tasks.workunit.client.0.trial144.stderr: "epoch_start": "23", 2026-02-19T13:18:07.475 INFO:tasks.workunit.client.0.trial144.stderr: "start": "MIN", 2026-02-19T13:18:07.475 INFO:tasks.workunit.client.0.trial144.stderr: "end": "MIN", 2026-02-19T13:18:07.475 INFO:tasks.workunit.client.0.trial144.stderr: "max_end": "MIN", 2026-02-19T13:18:07.475 INFO:tasks.workunit.client.0.trial144.stderr: "subset_last_update": "0'\''0", 2026-02-19T13:18:07.475 INFO:tasks.workunit.client.0.trial144.stderr: "deep": true, 2026-02-19T13:18:07.475 INFO:tasks.workunit.client.0.trial144.stderr: "req_scrub": false, 2026-02-19T13:18:07.475 INFO:tasks.workunit.client.0.trial144.stderr: "auto_repair": false, 2026-02-19T13:18:07.475 INFO:tasks.workunit.client.0.trial144.stderr: "check_repair": false, 2026-02-19T13:18:07.476 INFO:tasks.workunit.client.0.trial144.stderr: "deep_scrub_on_error": false, 2026-02-19T13:18:07.476 INFO:tasks.workunit.client.0.trial144.stderr: "priority": 5, 2026-02-19T13:18:07.476 INFO:tasks.workunit.client.0.trial144.stderr: "shallow_errors": 0, 2026-02-19T13:18:07.476 INFO:tasks.workunit.client.0.trial144.stderr: "deep_errors": 0, 2026-02-19T13:18:07.476 INFO:tasks.workunit.client.0.trial144.stderr: "fixed": 0, 2026-02-19T13:18:07.476 INFO:tasks.workunit.client.0.trial144.stderr: "waiting_on_whom": [], 2026-02-19T13:18:07.476 INFO:tasks.workunit.client.0.trial144.stderr: "schedule": "scrubbing", 2026-02-19T13:18:07.476 INFO:tasks.workunit.client.0.trial144.stderr: "is_reserving_replicas": true, 2026-02-19T13:18:07.476 INFO:tasks.workunit.client.0.trial144.stderr: "osd_to_respond": 4, 2026-02-19T13:18:07.476 INFO:tasks.workunit.client.0.trial144.stderr: "duration_seconds": 1, 2026-02-19T13:18:07.476 INFO:tasks.workunit.client.0.trial144.stderr: "requested_in_order": 2, 2026-02-19T13:18:07.477 INFO:tasks.workunit.client.0.trial144.stderr: "num_to_reserve": 2, 2026-02-19T13:18:07.477 INFO:tasks.workunit.client.0.trial144.stderr: "test_sequence": 9314307 2026-02-19T13:18:07.477 INFO:tasks.workunit.client.0.trial144.stderr:}' 2026-02-19T13:18:07.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:889: TEST_abort_periodic_for_operator: local pg2_is_reserving 2026-02-19T13:18:07.477 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:890: TEST_abort_periodic_for_operator: jq .is_reserving_replicas 2026-02-19T13:18:07.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:890: TEST_abort_periodic_for_operator: echo '{' '"active":' true, '"epoch_start":' '"23",' '"start":' '"MIN",' '"end":' '"MIN",' '"max_end":' '"MIN",' '"subset_last_update":' '"0'\''0",' '"deep":' true, '"req_scrub":' false, '"auto_repair":' false, '"check_repair":' false, '"deep_scrub_on_error":' false, '"priority":' 5, '"shallow_errors":' 0, '"deep_errors":' 0, '"fixed":' 0, '"waiting_on_whom":' '[],' '"schedule":' '"scrubbing",' '"is_reserving_replicas":' true, '"osd_to_respond":' 4, '"duration_seconds":' 1, '"requested_in_order":' 2, '"num_to_reserve":' 2, '"test_sequence":' 9314307 '}' 2026-02-19T13:18:07.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:890: TEST_abort_periodic_for_operator: pg2_is_reserving=true 2026-02-19T13:18:07.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:891: TEST_abort_periodic_for_operator: [[ true != \t\r\u\e ]] 2026-02-19T13:18:07.487 INFO:tasks.workunit.client.0.trial144.stdout:Instructing 1.9 to perform a high-priority scrub 2026-02-19T13:18:07.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:899: TEST_abort_periodic_for_operator: echo 'Instructing 1.9 to perform a high-priority scrub' 2026-02-19T13:18:07.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:900: TEST_abort_periodic_for_operator: ceph tell 1.9 scrub 2026-02-19T13:18:07.549 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:18:07.549 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:18:07.549 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:18:07.549 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:18:07.549 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:07.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:901: TEST_abort_periodic_for_operator: seq 1 10 2026-02-19T13:18:07.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:901: TEST_abort_periodic_for_operator: for i in $( seq 1 10 ) 2026-02-19T13:18:07.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:903: TEST_abort_periodic_for_operator: sleep 0.5 2026-02-19T13:18:08.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:904: TEST_abort_periodic_for_operator: ceph pg 1.9 query 2026-02-19T13:18:08.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:904: TEST_abort_periodic_for_operator: jq .scrubber 2026-02-19T13:18:08.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:904: TEST_abort_periodic_for_operator: stt2='{ 2026-02-19T13:18:08.136 INFO:tasks.workunit.client.0.trial144.stderr: "active": true, 2026-02-19T13:18:08.136 INFO:tasks.workunit.client.0.trial144.stderr: "epoch_start": "23", 2026-02-19T13:18:08.136 INFO:tasks.workunit.client.0.trial144.stderr: "start": "1:90000000::::0", 2026-02-19T13:18:08.136 INFO:tasks.workunit.client.0.trial144.stderr: "end": "MIN", 2026-02-19T13:18:08.137 INFO:tasks.workunit.client.0.trial144.stderr: "max_end": "MIN", 2026-02-19T13:18:08.137 INFO:tasks.workunit.client.0.trial144.stderr: "subset_last_update": "0'\''0", 2026-02-19T13:18:08.137 INFO:tasks.workunit.client.0.trial144.stderr: "deep": false, 2026-02-19T13:18:08.137 INFO:tasks.workunit.client.0.trial144.stderr: "req_scrub": true, 2026-02-19T13:18:08.137 INFO:tasks.workunit.client.0.trial144.stderr: "auto_repair": false, 2026-02-19T13:18:08.137 INFO:tasks.workunit.client.0.trial144.stderr: "check_repair": false, 2026-02-19T13:18:08.137 INFO:tasks.workunit.client.0.trial144.stderr: "deep_scrub_on_error": false, 2026-02-19T13:18:08.137 INFO:tasks.workunit.client.0.trial144.stderr: "priority": 5, 2026-02-19T13:18:08.137 INFO:tasks.workunit.client.0.trial144.stderr: "shallow_errors": 0, 2026-02-19T13:18:08.137 INFO:tasks.workunit.client.0.trial144.stderr: "deep_errors": 0, 2026-02-19T13:18:08.137 INFO:tasks.workunit.client.0.trial144.stderr: "fixed": 0, 2026-02-19T13:18:08.137 INFO:tasks.workunit.client.0.trial144.stderr: "waiting_on_whom": [], 2026-02-19T13:18:08.137 INFO:tasks.workunit.client.0.trial144.stderr: "schedule": "scrubbing", 2026-02-19T13:18:08.138 INFO:tasks.workunit.client.0.trial144.stderr: "is_reserving_replicas": false, 2026-02-19T13:18:08.138 INFO:tasks.workunit.client.0.trial144.stderr: "test_sequence": 9314309 2026-02-19T13:18:08.138 INFO:tasks.workunit.client.0.trial144.stderr:}' 2026-02-19T13:18:08.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:905: TEST_abort_periodic_for_operator: echo '{' '"active":' true, '"epoch_start":' '"23",' '"start":' '"1:90000000::::0",' '"end":' '"MIN",' '"max_end":' '"MIN",' '"subset_last_update":' '"0'\''0",' '"deep":' false, '"req_scrub":' true, '"auto_repair":' false, '"check_repair":' false, '"deep_scrub_on_error":' false, '"priority":' 5, '"shallow_errors":' 0, '"deep_errors":' 0, '"fixed":' 0, '"waiting_on_whom":' '[],' '"schedule":' '"scrubbing",' '"is_reserving_replicas":' false, '"test_sequence":' 9314309 '}' 2026-02-19T13:18:08.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:905: TEST_abort_periodic_for_operator: jq .active 2026-02-19T13:18:08.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:905: TEST_abort_periodic_for_operator: pg2_is_active=true 2026-02-19T13:18:08.148 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:906: TEST_abort_periodic_for_operator: echo '{' '"active":' true, '"epoch_start":' '"23",' '"start":' '"1:90000000::::0",' '"end":' '"MIN",' '"max_end":' '"MIN",' '"subset_last_update":' '"0'\''0",' '"deep":' false, '"req_scrub":' true, '"auto_repair":' false, '"check_repair":' false, '"deep_scrub_on_error":' false, '"priority":' 5, '"shallow_errors":' 0, '"deep_errors":' 0, '"fixed":' 0, '"waiting_on_whom":' '[],' '"schedule":' '"scrubbing",' '"is_reserving_replicas":' false, '"test_sequence":' 9314309 '}' 2026-02-19T13:18:08.148 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:906: TEST_abort_periodic_for_operator: jq .is_reserving_replicas 2026-02-19T13:18:08.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:906: TEST_abort_periodic_for_operator: pg2_is_reserving=false 2026-02-19T13:18:08.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:907: TEST_abort_periodic_for_operator: [[ true = \t\r\u\e ]] 2026-02-19T13:18:08.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:907: TEST_abort_periodic_for_operator: [[ false != \t\r\u\e ]] 2026-02-19T13:18:08.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:908: TEST_abort_periodic_for_operator: break 2026-02-19T13:18:08.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:913: TEST_abort_periodic_for_operator: [[ true != \t\r\u\e ]] 2026-02-19T13:18:08.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:913: TEST_abort_periodic_for_operator: [[ false = \t\r\u\e ]] 2026-02-19T13:18:08.159 INFO:tasks.workunit.client.0.trial144.stdout:Done 2026-02-19T13:18:08.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:917: TEST_abort_periodic_for_operator: echo Done 2026-02-19T13:18:08.160 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Teardown Test TEST_abort_periodic_for_operator ------------------ 2026-02-19T13:18:08.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_abort_periodic_for_operator ------------------' 2026-02-19T13:18:08.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-19T13:18:08.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:18:08.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:18:08.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:18:08.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:18:08.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:18:08.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:18:08.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:18:08.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:18:08.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:18:08.472 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:18:08.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:18:08.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:18:08.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:18:08.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:18:08.477 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:18:08.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:18:08.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:18:08.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:18:08.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:18:08.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:18:08.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:18:08.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:18:08.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:18:08.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:08.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:18:08.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:18:08.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:18:08.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:18:08.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_abort_periodic_for_operator ------------------' 2026-02-19T13:18:08.507 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Complete Test TEST_abort_periodic_for_operator ------------------ 2026-02-19T13:18:08.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-19T13:18:08.507 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Prepare Test TEST_deep_scrub_abort ------------------- 2026-02-19T13:18:08.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_deep_scrub_abort -------------------' 2026-02-19T13:18:08.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-19T13:18:08.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-test 2026-02-19T13:18:08.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-test 2026-02-19T13:18:08.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:18:08.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:18:08.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:18:08.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:18:08.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:18:08.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:18:08.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:18:08.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:18:08.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:18:08.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:18:08.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:18:08.511 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:18:08.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:18:08.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:18:08.513 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:18:08.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:18:08.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:18:08.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:18:08.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:18:08.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:18:08.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:18:08.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:18:08.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:18:08.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:08.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:18:08.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:18:08.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:18:08.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:18:08.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-test 2026-02-19T13:18:08.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:18:08.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:08.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:18:08.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.511299 2026-02-19T13:18:08.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:18:08.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:18:08.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:18:08.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-19T13:18:08.524 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Run Test TEST_deep_scrub_abort ----------------------- 2026-02-19T13:18:08.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_deep_scrub_abort -----------------------' 2026-02-19T13:18:08.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_deep_scrub_abort td/osd-scrub-test 2026-02-19T13:18:08.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:409: TEST_deep_scrub_abort: local dir=td/osd-scrub-test 2026-02-19T13:18:08.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:410: TEST_deep_scrub_abort: _scrub_abort td/osd-scrub-test deep-scrub 2026-02-19T13:18:08.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:293: _scrub_abort: local dir=td/osd-scrub-test 2026-02-19T13:18:08.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:294: _scrub_abort: local poolname=test 2026-02-19T13:18:08.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:295: _scrub_abort: local OSDS=3 2026-02-19T13:18:08.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:296: _scrub_abort: local objects=1000 2026-02-19T13:18:08.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:297: _scrub_abort: local type=deep-scrub 2026-02-19T13:18:08.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:299: _scrub_abort: TESTDATA=testdata.511299 2026-02-19T13:18:08.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:300: _scrub_abort: test deep-scrub = scrub 2026-02-19T13:18:08.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:305: _scrub_abort: stopscrub=nodeep-scrub 2026-02-19T13:18:08.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:306: _scrub_abort: check=nodeep_scrub 2026-02-19T13:18:08.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:309: _scrub_abort: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-19T13:18:08.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-test 2026-02-19T13:18:08.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:18:08.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:18:08.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:18:08.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-test/a 2026-02-19T13:18:08.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-19T13:18:08.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:18:08.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:08.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:08.546 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:08.546 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:08.547 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:18:08.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:18:08.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-19T13:18:08.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:18:08.575 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:18:08.575 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:18:08.575 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:18:08.575 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:18:08.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:18:08.576 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:18:08.576 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:18:08.577 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:18:08.577 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:18:08.577 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:08.577 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:18:08.577 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-mon.a.asok 2026-02-19T13:18:08.577 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:18:08.577 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.511299/ceph-mon.a.asok config get fsid 2026-02-19T13:18:08.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:18:08.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:18:08.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:18:08.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:18:08.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:18:08.633 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:18:08.633 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:18:08.634 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:18:08.634 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:18:08.634 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:08.634 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:18:08.634 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-mon.a.asok 2026-02-19T13:18:08.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:18:08.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.511299/ceph-mon.a.asok config get mon_host 2026-02-19T13:18:08.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:310: _scrub_abort: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-19T13:18:08.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-test 2026-02-19T13:18:08.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:18:08.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:18:08.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:18:08.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-test/x 2026-02-19T13:18:08.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:18:08.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:18:08.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:08.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:08.782 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:08.782 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:08.782 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:18:08.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:18:08.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:18:08.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-19T13:18:08.801 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:311: _scrub_abort: expr 3 - 1 2026-02-19T13:18:08.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:311: _scrub_abort: seq 0 2 2026-02-19T13:18:08.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:311: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:18:08.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:317: _scrub_abort: run_osd td/osd-scrub-test 0 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:08.804 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:08.805 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:18:08.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:18:08.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:18:08.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:18:08.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:18:08.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:18:08.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:18:08.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:18:08.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:18:08.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:18:08.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:18:08.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:18:08.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:18:08.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-19T13:18:08.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-19T13:18:08.806 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:18:08.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2916f810-154d-4d05-bb70-5e5143e08de8 2026-02-19T13:18:08.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 2916f810-154d-4d05-bb70-5e5143e08de8' 2026-02-19T13:18:08.807 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 2916f810-154d-4d05-bb70-5e5143e08de8 2026-02-19T13:18:08.807 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:18:08.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCQDZdpy4bTMBAAUxVLn+d+b7A5P+oFFj2d6w== 2026-02-19T13:18:08.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCQDZdpy4bTMBAAUxVLn+d+b7A5P+oFFj2d6w=="}' 2026-02-19T13:18:08.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2916f810-154d-4d05-bb70-5e5143e08de8 -i td/osd-scrub-test/0/new.json 2026-02-19T13:18:08.909 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:18:08.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-19T13:18:08.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQCQDZdpy4bTMBAAUxVLn+d+b7A5P+oFFj2d6w== --osd-uuid 2916f810-154d-4d05-bb70-5e5143e08de8 2026-02-19T13:18:08.934 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:08.929+0000 7f38ad685a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:08.936 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:08.933+0000 7f38ad685a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:08.937 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:08.933+0000 7f38ad685a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:08.937 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:08.933+0000 7f38ad685a40 -1 bdev(0x557ec1460c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:18:08.937 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:08.933+0000 7f38ad685a40 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-19T13:18:09.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-19T13:18:09.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:18:09.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:18:09.676 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:18:09.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:18:09.880 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:18:09.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:18:09.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-19T13:18:09.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:18:09.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:18:09.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:18:09.898 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:09.893+0000 7fdc3324da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:09.899 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:09.897+0000 7fdc3324da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:09.900 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:09.897+0000 7fdc3324da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:10.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:18:10.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:18:10.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:18:10.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:18:10.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:18:10.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:10.036 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:18:10.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:18:10.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:10.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:18:10.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:18:10.313 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:10.309+0000 7fdc3324da40 -1 Falling back to public interface 2026-02-19T13:18:10.944 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:10.941+0000 7fdc3324da40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:18:11.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:18:11.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:11.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:18:11.198 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:18:11.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:11.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:18:11.355 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/848209259,v1:127.0.0.1:6803/848209259] [v2:127.0.0.1:6804/848209259,v1:127.0.0.1:6805/848209259] exists,up 2916f810-154d-4d05-bb70-5e5143e08de8 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:311: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:317: _scrub_abort: run_osd td/osd-scrub-test 1 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:18:11.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:18:11.357 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:18:11.357 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:11.357 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:11.357 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:11.357 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:11.357 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:18:11.357 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:18:11.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:18:11.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:18:11.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:18:11.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:18:11.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:18:11.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:18:11.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:18:11.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:18:11.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:18:11.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:18:11.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:18:11.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-19T13:18:11.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-19T13:18:11.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:18:11.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a022b53b-4668-47df-8d1f-f21eba1bca38 2026-02-19T13:18:11.360 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 a022b53b-4668-47df-8d1f-f21eba1bca38 2026-02-19T13:18:11.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 a022b53b-4668-47df-8d1f-f21eba1bca38' 2026-02-19T13:18:11.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:18:11.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCTDZdpUQc5FhAA9FJYN7BWZ1rPsSJLvUWZPA== 2026-02-19T13:18:11.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCTDZdpUQc5FhAA9FJYN7BWZ1rPsSJLvUWZPA=="}' 2026-02-19T13:18:11.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a022b53b-4668-47df-8d1f-f21eba1bca38 -i td/osd-scrub-test/1/new.json 2026-02-19T13:18:11.537 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:18:11.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-19T13:18:11.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQCTDZdpUQc5FhAA9FJYN7BWZ1rPsSJLvUWZPA== --osd-uuid a022b53b-4668-47df-8d1f-f21eba1bca38 2026-02-19T13:18:11.565 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:11.561+0000 7f05f3b8aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:11.566 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:11.561+0000 7f05f3b8aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:11.567 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:11.565+0000 7f05f3b8aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:11.568 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:11.565+0000 7f05f3b8aa40 -1 bdev(0x5621430e4c00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:18:11.568 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:11.565+0000 7f05f3b8aa40 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-19T13:18:12.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-19T13:18:12.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:18:12.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:18:12.183 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:18:12.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:18:12.381 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:18:12.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:18:12.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-19T13:18:12.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:18:12.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:18:12.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:18:12.406 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:12.401+0000 7f96e0821a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:12.407 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:12.405+0000 7f96e0821a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:12.408 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:12.405+0000 7f96e0821a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:12.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:18:12.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:18:12.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:18:12.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:18:12.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:18:12.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:12.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:18:12.540 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:18:12.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:12.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:18:12.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:18:12.785 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:12.781+0000 7f96e0821a40 -1 Falling back to public interface 2026-02-19T13:18:13.334 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:13.329+0000 7f96e0821a40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:18:13.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:18:13.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:13.701 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:18:13.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:18:13.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:13.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:18:13.857 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/569890237,v1:127.0.0.1:6811/569890237] [v2:127.0.0.1:6812/569890237,v1:127.0.0.1:6813/569890237] exists,up a022b53b-4668-47df-8d1f-f21eba1bca38 2026-02-19T13:18:13.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:18:13.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:18:13.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:18:13.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:311: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:18:13.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:317: _scrub_abort: run_osd td/osd-scrub-test 2 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-19T13:18:13.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:18:13.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:18:13.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:18:13.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:18:13.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-19T13:18:13.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:18:13.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:18:13.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:18:13.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:18:13.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-19T13:18:13.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-19T13:18:13.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:18:13.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:18:13.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:18:13.862 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:18:13.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:13.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:13.863 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:13.863 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:13.863 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:18:13.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:18:13.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:18:13.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:18:13.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:18:13.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:18:13.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:18:13.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:18:13.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:18:13.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:18:13.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:18:13.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:18:13.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:18:13.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-19T13:18:13.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-19T13:18:13.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:18:13.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f1ec7760-ea3c-45e7-8ff8-5d28ba771ea1 2026-02-19T13:18:13.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 f1ec7760-ea3c-45e7-8ff8-5d28ba771ea1' 2026-02-19T13:18:13.866 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 f1ec7760-ea3c-45e7-8ff8-5d28ba771ea1 2026-02-19T13:18:13.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:18:13.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCVDZdpQ7MLNBAAdzG9zQzXqvyqYBjH5Zx5QA== 2026-02-19T13:18:13.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCVDZdpQ7MLNBAAdzG9zQzXqvyqYBjH5Zx5QA=="}' 2026-02-19T13:18:13.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f1ec7760-ea3c-45e7-8ff8-5d28ba771ea1 -i td/osd-scrub-test/2/new.json 2026-02-19T13:18:14.025 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:18:14.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-19T13:18:14.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQCVDZdpQ7MLNBAAdzG9zQzXqvyqYBjH5Zx5QA== --osd-uuid f1ec7760-ea3c-45e7-8ff8-5d28ba771ea1 2026-02-19T13:18:14.053 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:14.049+0000 7f99b036fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:14.055 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:14.049+0000 7f99b036fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:14.056 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:14.053+0000 7f99b036fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:14.056 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:14.053+0000 7f99b036fa40 -1 bdev(0x562bb39a2c00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:18:14.057 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:14.053+0000 7f99b036fa40 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-19T13:18:14.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-19T13:18:14.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:18:14.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:18:14.684 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T13:18:14.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:18:14.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:18:14.889 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T13:18:14.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-19T13:18:14.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:18:14.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:18:14.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:18:14.915 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:14.909+0000 7f3e9686ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:14.916 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:14.913+0000 7f3e9686ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:14.917 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:14.913+0000 7f3e9686ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:15.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:18:15.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:18:15.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:18:15.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:18:15.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:18:15.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:15.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:18:15.051 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:18:15.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:15.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:18:15.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:18:15.297 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:15.293+0000 7f3e9686ea40 -1 Falling back to public interface 2026-02-19T13:18:15.845 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:18:15.841+0000 7f3e9686ea40 -1 osd.2 0 log_to_monitors true 2026-02-19T13:18:16.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:18:16.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:16.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:18:16.208 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:18:16.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:16.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:18:16.364 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2823335395,v1:127.0.0.1:6819/2823335395] [v2:127.0.0.1:6820/2823335395,v1:127.0.0.1:6821/2823335395] exists,up f1ec7760-ea3c-45e7-8ff8-5d28ba771ea1 2026-02-19T13:18:16.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:18:16.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:18:16.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:18:16.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:325: _scrub_abort: create_pool test 1 1 2026-02-19T13:18:16.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-19T13:18:16.574 INFO:tasks.workunit.client.0.trial144.stderr:pool 'test' created 2026-02-19T13:18:16.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:18:17.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:326: _scrub_abort: wait_for_clean 2026-02-19T13:18:17.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:18:17.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:18:17.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:18:17.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:18:17.588 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:18:17.588 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:18:17.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:18:17.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:18:17.588 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:18:17.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:18:17.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:18:17.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:18:17.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:18:17.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:17.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:17.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:17.850 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:18:17.850 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:18:17.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:17.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:17.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:17.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-19T13:18:17.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-19T13:18:17.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-19T13:18:17.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:17.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:17.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705669 2026-02-19T13:18:17.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705669 2026-02-19T13:18:17.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-38654705669' 2026-02-19T13:18:17.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:17.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:18.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574851 2026-02-19T13:18:18.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574851 2026-02-19T13:18:18.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-55834574851' 2026-02-19T13:18:18.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:18.070 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-19T13:18:18.071 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:18.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:18.072 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-19T13:18:18.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:18.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-19T13:18:18.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-19T13:18:18.074 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836487 2026-02-19T13:18:18.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:18.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T13:18:18.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:18:19.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:18:19.245 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:19.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836489 -lt 21474836487 2026-02-19T13:18:19.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:19.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705669 2026-02-19T13:18:19.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:19.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:19.407 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705669 2026-02-19T13:18:19.407 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:19.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705669 2026-02-19T13:18:19.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-19T13:18:19.409 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705669 2026-02-19T13:18:19.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:19.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705670 -lt 38654705669 2026-02-19T13:18:19.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:19.568 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574851 2026-02-19T13:18:19.568 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:19.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:19.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574851 2026-02-19T13:18:19.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:19.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574851 2026-02-19T13:18:19.572 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574851 2026-02-19T13:18:19.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-19T13:18:19.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:19.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574852 -lt 55834574851 2026-02-19T13:18:19.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:18:19.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:19.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:19.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:18:19.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:18:19.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:18:19.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:18:19.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:18:19.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:18:19.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:18:19.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:18:20.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:18:20.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:18:20.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:20.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:20.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:18:20.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:18:20.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:18:20.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:327: _scrub_abort: ceph osd dump 2026-02-19T13:18:20.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:327: _scrub_abort: grep '^pool.*['\'']test['\'']' 2026-02-19T13:18:20.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:327: _scrub_abort: awk '{ print $2 }' 2026-02-19T13:18:20.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:327: _scrub_abort: poolid=1 2026-02-19T13:18:20.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:329: _scrub_abort: dd if=/dev/urandom of=testdata.511299 bs=1032 count=1 2026-02-19T13:18:20.414 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:18:20.414 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:18:20.414 INFO:tasks.workunit.client.0.trial144.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.6928e-05 s, 27.9 MB/s 2026-02-19T13:18:20.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: seq 1 1000 2026-02-19T13:18:20.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj1 testdata.511299 2026-02-19T13:18:20.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj2 testdata.511299 2026-02-19T13:18:20.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj3 testdata.511299 2026-02-19T13:18:20.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj4 testdata.511299 2026-02-19T13:18:20.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj5 testdata.511299 2026-02-19T13:18:20.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj6 testdata.511299 2026-02-19T13:18:20.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj7 testdata.511299 2026-02-19T13:18:20.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj8 testdata.511299 2026-02-19T13:18:20.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj9 testdata.511299 2026-02-19T13:18:20.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj10 testdata.511299 2026-02-19T13:18:20.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj11 testdata.511299 2026-02-19T13:18:20.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj12 testdata.511299 2026-02-19T13:18:20.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj13 testdata.511299 2026-02-19T13:18:20.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj14 testdata.511299 2026-02-19T13:18:20.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj15 testdata.511299 2026-02-19T13:18:20.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj16 testdata.511299 2026-02-19T13:18:20.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj17 testdata.511299 2026-02-19T13:18:20.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj18 testdata.511299 2026-02-19T13:18:20.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj19 testdata.511299 2026-02-19T13:18:20.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj20 testdata.511299 2026-02-19T13:18:20.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj21 testdata.511299 2026-02-19T13:18:20.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj22 testdata.511299 2026-02-19T13:18:20.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj23 testdata.511299 2026-02-19T13:18:20.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj24 testdata.511299 2026-02-19T13:18:20.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:20.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj25 testdata.511299 2026-02-19T13:18:21.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj26 testdata.511299 2026-02-19T13:18:21.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj27 testdata.511299 2026-02-19T13:18:21.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj28 testdata.511299 2026-02-19T13:18:21.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj29 testdata.511299 2026-02-19T13:18:21.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj30 testdata.511299 2026-02-19T13:18:21.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj31 testdata.511299 2026-02-19T13:18:21.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj32 testdata.511299 2026-02-19T13:18:21.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj33 testdata.511299 2026-02-19T13:18:21.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj34 testdata.511299 2026-02-19T13:18:21.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj35 testdata.511299 2026-02-19T13:18:21.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj36 testdata.511299 2026-02-19T13:18:21.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj37 testdata.511299 2026-02-19T13:18:21.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj38 testdata.511299 2026-02-19T13:18:21.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj39 testdata.511299 2026-02-19T13:18:21.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj40 testdata.511299 2026-02-19T13:18:21.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj41 testdata.511299 2026-02-19T13:18:21.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj42 testdata.511299 2026-02-19T13:18:21.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj43 testdata.511299 2026-02-19T13:18:21.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj44 testdata.511299 2026-02-19T13:18:21.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj45 testdata.511299 2026-02-19T13:18:21.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj46 testdata.511299 2026-02-19T13:18:21.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj47 testdata.511299 2026-02-19T13:18:21.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj48 testdata.511299 2026-02-19T13:18:21.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj49 testdata.511299 2026-02-19T13:18:21.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj50 testdata.511299 2026-02-19T13:18:21.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj51 testdata.511299 2026-02-19T13:18:21.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj52 testdata.511299 2026-02-19T13:18:21.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj53 testdata.511299 2026-02-19T13:18:21.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj54 testdata.511299 2026-02-19T13:18:21.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj55 testdata.511299 2026-02-19T13:18:21.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj56 testdata.511299 2026-02-19T13:18:21.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj57 testdata.511299 2026-02-19T13:18:21.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj58 testdata.511299 2026-02-19T13:18:21.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj59 testdata.511299 2026-02-19T13:18:21.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj60 testdata.511299 2026-02-19T13:18:21.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj61 testdata.511299 2026-02-19T13:18:21.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj62 testdata.511299 2026-02-19T13:18:21.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj63 testdata.511299 2026-02-19T13:18:21.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj64 testdata.511299 2026-02-19T13:18:21.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj65 testdata.511299 2026-02-19T13:18:21.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj66 testdata.511299 2026-02-19T13:18:21.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj67 testdata.511299 2026-02-19T13:18:21.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:21.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj68 testdata.511299 2026-02-19T13:18:22.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj69 testdata.511299 2026-02-19T13:18:22.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj70 testdata.511299 2026-02-19T13:18:22.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj71 testdata.511299 2026-02-19T13:18:22.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj72 testdata.511299 2026-02-19T13:18:22.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj73 testdata.511299 2026-02-19T13:18:22.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj74 testdata.511299 2026-02-19T13:18:22.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj75 testdata.511299 2026-02-19T13:18:22.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj76 testdata.511299 2026-02-19T13:18:22.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj77 testdata.511299 2026-02-19T13:18:22.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj78 testdata.511299 2026-02-19T13:18:22.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj79 testdata.511299 2026-02-19T13:18:22.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj80 testdata.511299 2026-02-19T13:18:22.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj81 testdata.511299 2026-02-19T13:18:22.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj82 testdata.511299 2026-02-19T13:18:22.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj83 testdata.511299 2026-02-19T13:18:22.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj84 testdata.511299 2026-02-19T13:18:22.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj85 testdata.511299 2026-02-19T13:18:22.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj86 testdata.511299 2026-02-19T13:18:22.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj87 testdata.511299 2026-02-19T13:18:22.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj88 testdata.511299 2026-02-19T13:18:22.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj89 testdata.511299 2026-02-19T13:18:22.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj90 testdata.511299 2026-02-19T13:18:22.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj91 testdata.511299 2026-02-19T13:18:22.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj92 testdata.511299 2026-02-19T13:18:22.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj93 testdata.511299 2026-02-19T13:18:22.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj94 testdata.511299 2026-02-19T13:18:22.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj95 testdata.511299 2026-02-19T13:18:22.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj96 testdata.511299 2026-02-19T13:18:22.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj97 testdata.511299 2026-02-19T13:18:22.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj98 testdata.511299 2026-02-19T13:18:22.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj99 testdata.511299 2026-02-19T13:18:22.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj100 testdata.511299 2026-02-19T13:18:22.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj101 testdata.511299 2026-02-19T13:18:22.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj102 testdata.511299 2026-02-19T13:18:22.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj103 testdata.511299 2026-02-19T13:18:22.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj104 testdata.511299 2026-02-19T13:18:22.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj105 testdata.511299 2026-02-19T13:18:22.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj106 testdata.511299 2026-02-19T13:18:22.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj107 testdata.511299 2026-02-19T13:18:22.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj108 testdata.511299 2026-02-19T13:18:22.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj109 testdata.511299 2026-02-19T13:18:22.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:22.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj110 testdata.511299 2026-02-19T13:18:23.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj111 testdata.511299 2026-02-19T13:18:23.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj112 testdata.511299 2026-02-19T13:18:23.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj113 testdata.511299 2026-02-19T13:18:23.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj114 testdata.511299 2026-02-19T13:18:23.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj115 testdata.511299 2026-02-19T13:18:23.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj116 testdata.511299 2026-02-19T13:18:23.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj117 testdata.511299 2026-02-19T13:18:23.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj118 testdata.511299 2026-02-19T13:18:23.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj119 testdata.511299 2026-02-19T13:18:23.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj120 testdata.511299 2026-02-19T13:18:23.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj121 testdata.511299 2026-02-19T13:18:23.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj122 testdata.511299 2026-02-19T13:18:23.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj123 testdata.511299 2026-02-19T13:18:23.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj124 testdata.511299 2026-02-19T13:18:23.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj125 testdata.511299 2026-02-19T13:18:23.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj126 testdata.511299 2026-02-19T13:18:23.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj127 testdata.511299 2026-02-19T13:18:23.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj128 testdata.511299 2026-02-19T13:18:23.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj129 testdata.511299 2026-02-19T13:18:23.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj130 testdata.511299 2026-02-19T13:18:23.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj131 testdata.511299 2026-02-19T13:18:23.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj132 testdata.511299 2026-02-19T13:18:23.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj133 testdata.511299 2026-02-19T13:18:23.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj134 testdata.511299 2026-02-19T13:18:23.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj135 testdata.511299 2026-02-19T13:18:23.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj136 testdata.511299 2026-02-19T13:18:23.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj137 testdata.511299 2026-02-19T13:18:23.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj138 testdata.511299 2026-02-19T13:18:23.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj139 testdata.511299 2026-02-19T13:18:23.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj140 testdata.511299 2026-02-19T13:18:23.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj141 testdata.511299 2026-02-19T13:18:23.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj142 testdata.511299 2026-02-19T13:18:23.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj143 testdata.511299 2026-02-19T13:18:23.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj144 testdata.511299 2026-02-19T13:18:23.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj145 testdata.511299 2026-02-19T13:18:23.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj146 testdata.511299 2026-02-19T13:18:23.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj147 testdata.511299 2026-02-19T13:18:23.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj148 testdata.511299 2026-02-19T13:18:23.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj149 testdata.511299 2026-02-19T13:18:23.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj150 testdata.511299 2026-02-19T13:18:23.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj151 testdata.511299 2026-02-19T13:18:23.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj152 testdata.511299 2026-02-19T13:18:23.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:23.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj153 testdata.511299 2026-02-19T13:18:24.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj154 testdata.511299 2026-02-19T13:18:24.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj155 testdata.511299 2026-02-19T13:18:24.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj156 testdata.511299 2026-02-19T13:18:24.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj157 testdata.511299 2026-02-19T13:18:24.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj158 testdata.511299 2026-02-19T13:18:24.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj159 testdata.511299 2026-02-19T13:18:24.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj160 testdata.511299 2026-02-19T13:18:24.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj161 testdata.511299 2026-02-19T13:18:24.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj162 testdata.511299 2026-02-19T13:18:24.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj163 testdata.511299 2026-02-19T13:18:24.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj164 testdata.511299 2026-02-19T13:18:24.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj165 testdata.511299 2026-02-19T13:18:24.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj166 testdata.511299 2026-02-19T13:18:24.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj167 testdata.511299 2026-02-19T13:18:24.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj168 testdata.511299 2026-02-19T13:18:24.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj169 testdata.511299 2026-02-19T13:18:24.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj170 testdata.511299 2026-02-19T13:18:24.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj171 testdata.511299 2026-02-19T13:18:24.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj172 testdata.511299 2026-02-19T13:18:24.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj173 testdata.511299 2026-02-19T13:18:24.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj174 testdata.511299 2026-02-19T13:18:24.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj175 testdata.511299 2026-02-19T13:18:24.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj176 testdata.511299 2026-02-19T13:18:24.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj177 testdata.511299 2026-02-19T13:18:24.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj178 testdata.511299 2026-02-19T13:18:24.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj179 testdata.511299 2026-02-19T13:18:24.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj180 testdata.511299 2026-02-19T13:18:24.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj181 testdata.511299 2026-02-19T13:18:24.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj182 testdata.511299 2026-02-19T13:18:24.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj183 testdata.511299 2026-02-19T13:18:24.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj184 testdata.511299 2026-02-19T13:18:24.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj185 testdata.511299 2026-02-19T13:18:24.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj186 testdata.511299 2026-02-19T13:18:24.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj187 testdata.511299 2026-02-19T13:18:24.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj188 testdata.511299 2026-02-19T13:18:24.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj189 testdata.511299 2026-02-19T13:18:24.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj190 testdata.511299 2026-02-19T13:18:24.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj191 testdata.511299 2026-02-19T13:18:24.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj192 testdata.511299 2026-02-19T13:18:24.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj193 testdata.511299 2026-02-19T13:18:24.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj194 testdata.511299 2026-02-19T13:18:24.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj195 testdata.511299 2026-02-19T13:18:24.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj196 testdata.511299 2026-02-19T13:18:24.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:24.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj197 testdata.511299 2026-02-19T13:18:25.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj198 testdata.511299 2026-02-19T13:18:25.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj199 testdata.511299 2026-02-19T13:18:25.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj200 testdata.511299 2026-02-19T13:18:25.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj201 testdata.511299 2026-02-19T13:18:25.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj202 testdata.511299 2026-02-19T13:18:25.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj203 testdata.511299 2026-02-19T13:18:25.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj204 testdata.511299 2026-02-19T13:18:25.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj205 testdata.511299 2026-02-19T13:18:25.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj206 testdata.511299 2026-02-19T13:18:25.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj207 testdata.511299 2026-02-19T13:18:25.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj208 testdata.511299 2026-02-19T13:18:25.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj209 testdata.511299 2026-02-19T13:18:25.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj210 testdata.511299 2026-02-19T13:18:25.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj211 testdata.511299 2026-02-19T13:18:25.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj212 testdata.511299 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj213 testdata.511299 2026-02-19T13:18:25.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj214 testdata.511299 2026-02-19T13:18:25.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj215 testdata.511299 2026-02-19T13:18:25.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj216 testdata.511299 2026-02-19T13:18:25.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj217 testdata.511299 2026-02-19T13:18:25.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj218 testdata.511299 2026-02-19T13:18:25.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj219 testdata.511299 2026-02-19T13:18:25.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj220 testdata.511299 2026-02-19T13:18:25.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj221 testdata.511299 2026-02-19T13:18:25.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj222 testdata.511299 2026-02-19T13:18:25.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj223 testdata.511299 2026-02-19T13:18:25.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj224 testdata.511299 2026-02-19T13:18:25.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj225 testdata.511299 2026-02-19T13:18:25.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj226 testdata.511299 2026-02-19T13:18:25.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj227 testdata.511299 2026-02-19T13:18:25.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj228 testdata.511299 2026-02-19T13:18:25.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj229 testdata.511299 2026-02-19T13:18:25.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj230 testdata.511299 2026-02-19T13:18:25.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj231 testdata.511299 2026-02-19T13:18:25.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj232 testdata.511299 2026-02-19T13:18:25.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj233 testdata.511299 2026-02-19T13:18:25.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj234 testdata.511299 2026-02-19T13:18:25.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj235 testdata.511299 2026-02-19T13:18:25.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj236 testdata.511299 2026-02-19T13:18:25.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj237 testdata.511299 2026-02-19T13:18:25.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj238 testdata.511299 2026-02-19T13:18:25.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj239 testdata.511299 2026-02-19T13:18:25.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj240 testdata.511299 2026-02-19T13:18:25.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:25.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj241 testdata.511299 2026-02-19T13:18:26.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj242 testdata.511299 2026-02-19T13:18:26.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj243 testdata.511299 2026-02-19T13:18:26.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj244 testdata.511299 2026-02-19T13:18:26.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj245 testdata.511299 2026-02-19T13:18:26.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj246 testdata.511299 2026-02-19T13:18:26.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj247 testdata.511299 2026-02-19T13:18:26.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj248 testdata.511299 2026-02-19T13:18:26.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj249 testdata.511299 2026-02-19T13:18:26.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj250 testdata.511299 2026-02-19T13:18:26.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj251 testdata.511299 2026-02-19T13:18:26.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj252 testdata.511299 2026-02-19T13:18:26.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj253 testdata.511299 2026-02-19T13:18:26.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj254 testdata.511299 2026-02-19T13:18:26.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj255 testdata.511299 2026-02-19T13:18:26.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj256 testdata.511299 2026-02-19T13:18:26.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj257 testdata.511299 2026-02-19T13:18:26.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj258 testdata.511299 2026-02-19T13:18:26.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj259 testdata.511299 2026-02-19T13:18:26.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj260 testdata.511299 2026-02-19T13:18:26.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj261 testdata.511299 2026-02-19T13:18:26.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj262 testdata.511299 2026-02-19T13:18:26.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj263 testdata.511299 2026-02-19T13:18:26.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj264 testdata.511299 2026-02-19T13:18:26.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj265 testdata.511299 2026-02-19T13:18:26.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj266 testdata.511299 2026-02-19T13:18:26.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj267 testdata.511299 2026-02-19T13:18:26.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj268 testdata.511299 2026-02-19T13:18:26.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj269 testdata.511299 2026-02-19T13:18:26.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj270 testdata.511299 2026-02-19T13:18:26.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj271 testdata.511299 2026-02-19T13:18:26.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj272 testdata.511299 2026-02-19T13:18:26.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj273 testdata.511299 2026-02-19T13:18:26.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj274 testdata.511299 2026-02-19T13:18:26.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj275 testdata.511299 2026-02-19T13:18:26.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj276 testdata.511299 2026-02-19T13:18:26.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj277 testdata.511299 2026-02-19T13:18:26.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj278 testdata.511299 2026-02-19T13:18:26.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj279 testdata.511299 2026-02-19T13:18:26.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj280 testdata.511299 2026-02-19T13:18:26.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj281 testdata.511299 2026-02-19T13:18:26.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj282 testdata.511299 2026-02-19T13:18:26.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:26.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj283 testdata.511299 2026-02-19T13:18:27.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj284 testdata.511299 2026-02-19T13:18:27.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj285 testdata.511299 2026-02-19T13:18:27.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj286 testdata.511299 2026-02-19T13:18:27.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj287 testdata.511299 2026-02-19T13:18:27.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj288 testdata.511299 2026-02-19T13:18:27.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj289 testdata.511299 2026-02-19T13:18:27.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj290 testdata.511299 2026-02-19T13:18:27.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj291 testdata.511299 2026-02-19T13:18:27.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj292 testdata.511299 2026-02-19T13:18:27.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj293 testdata.511299 2026-02-19T13:18:27.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj294 testdata.511299 2026-02-19T13:18:27.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj295 testdata.511299 2026-02-19T13:18:27.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj296 testdata.511299 2026-02-19T13:18:27.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj297 testdata.511299 2026-02-19T13:18:27.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj298 testdata.511299 2026-02-19T13:18:27.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj299 testdata.511299 2026-02-19T13:18:27.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj300 testdata.511299 2026-02-19T13:18:27.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj301 testdata.511299 2026-02-19T13:18:27.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj302 testdata.511299 2026-02-19T13:18:27.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj303 testdata.511299 2026-02-19T13:18:27.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj304 testdata.511299 2026-02-19T13:18:27.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj305 testdata.511299 2026-02-19T13:18:27.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj306 testdata.511299 2026-02-19T13:18:27.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj307 testdata.511299 2026-02-19T13:18:27.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj308 testdata.511299 2026-02-19T13:18:27.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj309 testdata.511299 2026-02-19T13:18:27.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj310 testdata.511299 2026-02-19T13:18:27.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj311 testdata.511299 2026-02-19T13:18:27.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj312 testdata.511299 2026-02-19T13:18:27.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj313 testdata.511299 2026-02-19T13:18:27.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj314 testdata.511299 2026-02-19T13:18:27.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj315 testdata.511299 2026-02-19T13:18:27.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj316 testdata.511299 2026-02-19T13:18:27.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj317 testdata.511299 2026-02-19T13:18:27.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj318 testdata.511299 2026-02-19T13:18:27.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj319 testdata.511299 2026-02-19T13:18:27.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj320 testdata.511299 2026-02-19T13:18:27.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj321 testdata.511299 2026-02-19T13:18:27.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj322 testdata.511299 2026-02-19T13:18:27.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj323 testdata.511299 2026-02-19T13:18:27.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj324 testdata.511299 2026-02-19T13:18:27.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj325 testdata.511299 2026-02-19T13:18:27.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:27.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj326 testdata.511299 2026-02-19T13:18:28.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj327 testdata.511299 2026-02-19T13:18:28.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj328 testdata.511299 2026-02-19T13:18:28.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj329 testdata.511299 2026-02-19T13:18:28.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj330 testdata.511299 2026-02-19T13:18:28.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj331 testdata.511299 2026-02-19T13:18:28.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj332 testdata.511299 2026-02-19T13:18:28.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj333 testdata.511299 2026-02-19T13:18:28.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj334 testdata.511299 2026-02-19T13:18:28.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj335 testdata.511299 2026-02-19T13:18:28.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj336 testdata.511299 2026-02-19T13:18:28.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj337 testdata.511299 2026-02-19T13:18:28.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj338 testdata.511299 2026-02-19T13:18:28.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj339 testdata.511299 2026-02-19T13:18:28.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj340 testdata.511299 2026-02-19T13:18:28.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj341 testdata.511299 2026-02-19T13:18:28.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj342 testdata.511299 2026-02-19T13:18:28.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj343 testdata.511299 2026-02-19T13:18:28.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj344 testdata.511299 2026-02-19T13:18:28.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj345 testdata.511299 2026-02-19T13:18:28.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj346 testdata.511299 2026-02-19T13:18:28.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj347 testdata.511299 2026-02-19T13:18:28.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj348 testdata.511299 2026-02-19T13:18:28.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj349 testdata.511299 2026-02-19T13:18:28.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj350 testdata.511299 2026-02-19T13:18:28.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj351 testdata.511299 2026-02-19T13:18:28.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj352 testdata.511299 2026-02-19T13:18:28.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj353 testdata.511299 2026-02-19T13:18:28.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj354 testdata.511299 2026-02-19T13:18:28.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj355 testdata.511299 2026-02-19T13:18:28.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj356 testdata.511299 2026-02-19T13:18:28.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj357 testdata.511299 2026-02-19T13:18:28.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj358 testdata.511299 2026-02-19T13:18:28.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj359 testdata.511299 2026-02-19T13:18:28.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj360 testdata.511299 2026-02-19T13:18:28.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj361 testdata.511299 2026-02-19T13:18:28.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj362 testdata.511299 2026-02-19T13:18:28.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj363 testdata.511299 2026-02-19T13:18:28.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj364 testdata.511299 2026-02-19T13:18:28.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj365 testdata.511299 2026-02-19T13:18:28.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj366 testdata.511299 2026-02-19T13:18:28.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj367 testdata.511299 2026-02-19T13:18:28.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj368 testdata.511299 2026-02-19T13:18:28.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:28.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj369 testdata.511299 2026-02-19T13:18:29.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj370 testdata.511299 2026-02-19T13:18:29.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj371 testdata.511299 2026-02-19T13:18:29.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj372 testdata.511299 2026-02-19T13:18:29.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj373 testdata.511299 2026-02-19T13:18:29.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj374 testdata.511299 2026-02-19T13:18:29.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj375 testdata.511299 2026-02-19T13:18:29.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj376 testdata.511299 2026-02-19T13:18:29.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj377 testdata.511299 2026-02-19T13:18:29.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj378 testdata.511299 2026-02-19T13:18:29.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj379 testdata.511299 2026-02-19T13:18:29.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj380 testdata.511299 2026-02-19T13:18:29.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj381 testdata.511299 2026-02-19T13:18:29.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj382 testdata.511299 2026-02-19T13:18:29.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj383 testdata.511299 2026-02-19T13:18:29.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj384 testdata.511299 2026-02-19T13:18:29.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj385 testdata.511299 2026-02-19T13:18:29.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj386 testdata.511299 2026-02-19T13:18:29.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj387 testdata.511299 2026-02-19T13:18:29.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj388 testdata.511299 2026-02-19T13:18:29.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj389 testdata.511299 2026-02-19T13:18:29.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj390 testdata.511299 2026-02-19T13:18:29.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj391 testdata.511299 2026-02-19T13:18:29.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj392 testdata.511299 2026-02-19T13:18:29.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj393 testdata.511299 2026-02-19T13:18:29.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj394 testdata.511299 2026-02-19T13:18:29.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj395 testdata.511299 2026-02-19T13:18:29.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj396 testdata.511299 2026-02-19T13:18:29.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj397 testdata.511299 2026-02-19T13:18:29.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj398 testdata.511299 2026-02-19T13:18:29.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj399 testdata.511299 2026-02-19T13:18:29.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj400 testdata.511299 2026-02-19T13:18:29.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj401 testdata.511299 2026-02-19T13:18:29.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj402 testdata.511299 2026-02-19T13:18:29.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj403 testdata.511299 2026-02-19T13:18:29.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj404 testdata.511299 2026-02-19T13:18:29.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj405 testdata.511299 2026-02-19T13:18:29.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj406 testdata.511299 2026-02-19T13:18:29.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj407 testdata.511299 2026-02-19T13:18:29.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj408 testdata.511299 2026-02-19T13:18:29.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj409 testdata.511299 2026-02-19T13:18:29.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj410 testdata.511299 2026-02-19T13:18:29.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj411 testdata.511299 2026-02-19T13:18:29.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj412 testdata.511299 2026-02-19T13:18:29.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:29.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj413 testdata.511299 2026-02-19T13:18:30.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj414 testdata.511299 2026-02-19T13:18:30.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj415 testdata.511299 2026-02-19T13:18:30.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj416 testdata.511299 2026-02-19T13:18:30.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj417 testdata.511299 2026-02-19T13:18:30.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj418 testdata.511299 2026-02-19T13:18:30.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj419 testdata.511299 2026-02-19T13:18:30.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj420 testdata.511299 2026-02-19T13:18:30.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj421 testdata.511299 2026-02-19T13:18:30.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj422 testdata.511299 2026-02-19T13:18:30.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj423 testdata.511299 2026-02-19T13:18:30.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj424 testdata.511299 2026-02-19T13:18:30.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj425 testdata.511299 2026-02-19T13:18:30.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj426 testdata.511299 2026-02-19T13:18:30.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj427 testdata.511299 2026-02-19T13:18:30.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj428 testdata.511299 2026-02-19T13:18:30.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj429 testdata.511299 2026-02-19T13:18:30.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj430 testdata.511299 2026-02-19T13:18:30.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj431 testdata.511299 2026-02-19T13:18:30.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj432 testdata.511299 2026-02-19T13:18:30.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj433 testdata.511299 2026-02-19T13:18:30.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj434 testdata.511299 2026-02-19T13:18:30.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj435 testdata.511299 2026-02-19T13:18:30.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj436 testdata.511299 2026-02-19T13:18:30.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj437 testdata.511299 2026-02-19T13:18:30.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj438 testdata.511299 2026-02-19T13:18:30.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj439 testdata.511299 2026-02-19T13:18:30.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj440 testdata.511299 2026-02-19T13:18:30.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj441 testdata.511299 2026-02-19T13:18:30.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj442 testdata.511299 2026-02-19T13:18:30.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj443 testdata.511299 2026-02-19T13:18:30.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj444 testdata.511299 2026-02-19T13:18:30.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj445 testdata.511299 2026-02-19T13:18:30.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj446 testdata.511299 2026-02-19T13:18:30.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj447 testdata.511299 2026-02-19T13:18:30.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj448 testdata.511299 2026-02-19T13:18:30.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj449 testdata.511299 2026-02-19T13:18:30.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj450 testdata.511299 2026-02-19T13:18:30.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj451 testdata.511299 2026-02-19T13:18:30.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj452 testdata.511299 2026-02-19T13:18:30.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj453 testdata.511299 2026-02-19T13:18:30.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj454 testdata.511299 2026-02-19T13:18:30.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj455 testdata.511299 2026-02-19T13:18:30.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:30.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj456 testdata.511299 2026-02-19T13:18:31.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj457 testdata.511299 2026-02-19T13:18:31.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj458 testdata.511299 2026-02-19T13:18:31.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj459 testdata.511299 2026-02-19T13:18:31.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj460 testdata.511299 2026-02-19T13:18:31.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj461 testdata.511299 2026-02-19T13:18:31.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj462 testdata.511299 2026-02-19T13:18:31.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj463 testdata.511299 2026-02-19T13:18:31.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj464 testdata.511299 2026-02-19T13:18:31.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj465 testdata.511299 2026-02-19T13:18:31.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj466 testdata.511299 2026-02-19T13:18:31.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj467 testdata.511299 2026-02-19T13:18:31.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj468 testdata.511299 2026-02-19T13:18:31.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj469 testdata.511299 2026-02-19T13:18:31.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj470 testdata.511299 2026-02-19T13:18:31.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj471 testdata.511299 2026-02-19T13:18:31.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj472 testdata.511299 2026-02-19T13:18:31.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj473 testdata.511299 2026-02-19T13:18:31.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj474 testdata.511299 2026-02-19T13:18:31.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj475 testdata.511299 2026-02-19T13:18:31.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj476 testdata.511299 2026-02-19T13:18:31.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj477 testdata.511299 2026-02-19T13:18:31.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj478 testdata.511299 2026-02-19T13:18:31.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj479 testdata.511299 2026-02-19T13:18:31.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj480 testdata.511299 2026-02-19T13:18:31.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj481 testdata.511299 2026-02-19T13:18:31.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj482 testdata.511299 2026-02-19T13:18:31.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj483 testdata.511299 2026-02-19T13:18:31.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj484 testdata.511299 2026-02-19T13:18:31.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj485 testdata.511299 2026-02-19T13:18:31.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj486 testdata.511299 2026-02-19T13:18:31.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj487 testdata.511299 2026-02-19T13:18:31.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj488 testdata.511299 2026-02-19T13:18:31.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj489 testdata.511299 2026-02-19T13:18:31.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj490 testdata.511299 2026-02-19T13:18:31.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj491 testdata.511299 2026-02-19T13:18:31.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj492 testdata.511299 2026-02-19T13:18:31.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj493 testdata.511299 2026-02-19T13:18:31.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj494 testdata.511299 2026-02-19T13:18:31.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj495 testdata.511299 2026-02-19T13:18:31.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj496 testdata.511299 2026-02-19T13:18:31.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj497 testdata.511299 2026-02-19T13:18:31.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:31.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj498 testdata.511299 2026-02-19T13:18:31.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj499 testdata.511299 2026-02-19T13:18:32.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj500 testdata.511299 2026-02-19T13:18:32.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj501 testdata.511299 2026-02-19T13:18:32.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj502 testdata.511299 2026-02-19T13:18:32.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj503 testdata.511299 2026-02-19T13:18:32.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj504 testdata.511299 2026-02-19T13:18:32.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj505 testdata.511299 2026-02-19T13:18:32.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj506 testdata.511299 2026-02-19T13:18:32.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj507 testdata.511299 2026-02-19T13:18:32.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj508 testdata.511299 2026-02-19T13:18:32.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj509 testdata.511299 2026-02-19T13:18:32.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj510 testdata.511299 2026-02-19T13:18:32.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj511 testdata.511299 2026-02-19T13:18:32.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj512 testdata.511299 2026-02-19T13:18:32.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj513 testdata.511299 2026-02-19T13:18:32.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj514 testdata.511299 2026-02-19T13:18:32.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj515 testdata.511299 2026-02-19T13:18:32.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj516 testdata.511299 2026-02-19T13:18:32.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj517 testdata.511299 2026-02-19T13:18:32.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj518 testdata.511299 2026-02-19T13:18:32.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj519 testdata.511299 2026-02-19T13:18:32.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj520 testdata.511299 2026-02-19T13:18:32.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj521 testdata.511299 2026-02-19T13:18:32.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj522 testdata.511299 2026-02-19T13:18:32.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj523 testdata.511299 2026-02-19T13:18:32.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj524 testdata.511299 2026-02-19T13:18:32.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj525 testdata.511299 2026-02-19T13:18:32.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj526 testdata.511299 2026-02-19T13:18:32.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj527 testdata.511299 2026-02-19T13:18:32.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj528 testdata.511299 2026-02-19T13:18:32.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj529 testdata.511299 2026-02-19T13:18:32.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj530 testdata.511299 2026-02-19T13:18:32.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj531 testdata.511299 2026-02-19T13:18:32.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj532 testdata.511299 2026-02-19T13:18:32.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj533 testdata.511299 2026-02-19T13:18:32.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj534 testdata.511299 2026-02-19T13:18:32.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj535 testdata.511299 2026-02-19T13:18:32.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj536 testdata.511299 2026-02-19T13:18:32.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj537 testdata.511299 2026-02-19T13:18:32.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj538 testdata.511299 2026-02-19T13:18:32.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj539 testdata.511299 2026-02-19T13:18:32.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj540 testdata.511299 2026-02-19T13:18:32.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj541 testdata.511299 2026-02-19T13:18:32.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:32.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj542 testdata.511299 2026-02-19T13:18:33.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj543 testdata.511299 2026-02-19T13:18:33.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj544 testdata.511299 2026-02-19T13:18:33.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj545 testdata.511299 2026-02-19T13:18:33.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj546 testdata.511299 2026-02-19T13:18:33.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj547 testdata.511299 2026-02-19T13:18:33.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj548 testdata.511299 2026-02-19T13:18:33.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj549 testdata.511299 2026-02-19T13:18:33.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj550 testdata.511299 2026-02-19T13:18:33.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj551 testdata.511299 2026-02-19T13:18:33.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj552 testdata.511299 2026-02-19T13:18:33.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj553 testdata.511299 2026-02-19T13:18:33.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj554 testdata.511299 2026-02-19T13:18:33.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj555 testdata.511299 2026-02-19T13:18:33.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj556 testdata.511299 2026-02-19T13:18:33.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj557 testdata.511299 2026-02-19T13:18:33.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj558 testdata.511299 2026-02-19T13:18:33.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj559 testdata.511299 2026-02-19T13:18:33.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj560 testdata.511299 2026-02-19T13:18:33.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj561 testdata.511299 2026-02-19T13:18:33.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj562 testdata.511299 2026-02-19T13:18:33.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj563 testdata.511299 2026-02-19T13:18:33.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj564 testdata.511299 2026-02-19T13:18:33.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj565 testdata.511299 2026-02-19T13:18:33.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj566 testdata.511299 2026-02-19T13:18:33.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj567 testdata.511299 2026-02-19T13:18:33.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj568 testdata.511299 2026-02-19T13:18:33.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj569 testdata.511299 2026-02-19T13:18:33.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj570 testdata.511299 2026-02-19T13:18:33.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj571 testdata.511299 2026-02-19T13:18:33.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj572 testdata.511299 2026-02-19T13:18:33.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj573 testdata.511299 2026-02-19T13:18:33.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj574 testdata.511299 2026-02-19T13:18:33.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj575 testdata.511299 2026-02-19T13:18:33.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj576 testdata.511299 2026-02-19T13:18:33.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj577 testdata.511299 2026-02-19T13:18:33.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj578 testdata.511299 2026-02-19T13:18:33.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj579 testdata.511299 2026-02-19T13:18:33.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj580 testdata.511299 2026-02-19T13:18:33.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj581 testdata.511299 2026-02-19T13:18:33.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj582 testdata.511299 2026-02-19T13:18:33.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj583 testdata.511299 2026-02-19T13:18:33.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj584 testdata.511299 2026-02-19T13:18:33.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:33.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj585 testdata.511299 2026-02-19T13:18:34.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj586 testdata.511299 2026-02-19T13:18:34.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj587 testdata.511299 2026-02-19T13:18:34.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj588 testdata.511299 2026-02-19T13:18:34.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj589 testdata.511299 2026-02-19T13:18:34.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj590 testdata.511299 2026-02-19T13:18:34.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj591 testdata.511299 2026-02-19T13:18:34.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj592 testdata.511299 2026-02-19T13:18:34.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj593 testdata.511299 2026-02-19T13:18:34.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj594 testdata.511299 2026-02-19T13:18:34.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj595 testdata.511299 2026-02-19T13:18:34.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj596 testdata.511299 2026-02-19T13:18:34.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj597 testdata.511299 2026-02-19T13:18:34.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj598 testdata.511299 2026-02-19T13:18:34.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj599 testdata.511299 2026-02-19T13:18:34.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj600 testdata.511299 2026-02-19T13:18:34.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj601 testdata.511299 2026-02-19T13:18:34.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj602 testdata.511299 2026-02-19T13:18:34.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj603 testdata.511299 2026-02-19T13:18:34.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj604 testdata.511299 2026-02-19T13:18:34.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj605 testdata.511299 2026-02-19T13:18:34.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj606 testdata.511299 2026-02-19T13:18:34.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj607 testdata.511299 2026-02-19T13:18:34.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj608 testdata.511299 2026-02-19T13:18:34.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj609 testdata.511299 2026-02-19T13:18:34.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj610 testdata.511299 2026-02-19T13:18:34.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj611 testdata.511299 2026-02-19T13:18:34.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj612 testdata.511299 2026-02-19T13:18:34.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj613 testdata.511299 2026-02-19T13:18:34.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj614 testdata.511299 2026-02-19T13:18:34.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj615 testdata.511299 2026-02-19T13:18:34.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj616 testdata.511299 2026-02-19T13:18:34.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj617 testdata.511299 2026-02-19T13:18:34.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj618 testdata.511299 2026-02-19T13:18:34.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj619 testdata.511299 2026-02-19T13:18:34.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj620 testdata.511299 2026-02-19T13:18:34.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj621 testdata.511299 2026-02-19T13:18:34.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj622 testdata.511299 2026-02-19T13:18:34.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj623 testdata.511299 2026-02-19T13:18:34.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj624 testdata.511299 2026-02-19T13:18:34.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj625 testdata.511299 2026-02-19T13:18:34.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj626 testdata.511299 2026-02-19T13:18:34.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj627 testdata.511299 2026-02-19T13:18:34.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj628 testdata.511299 2026-02-19T13:18:34.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:34.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj629 testdata.511299 2026-02-19T13:18:35.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj630 testdata.511299 2026-02-19T13:18:35.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj631 testdata.511299 2026-02-19T13:18:35.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj632 testdata.511299 2026-02-19T13:18:35.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj633 testdata.511299 2026-02-19T13:18:35.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj634 testdata.511299 2026-02-19T13:18:35.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj635 testdata.511299 2026-02-19T13:18:35.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj636 testdata.511299 2026-02-19T13:18:35.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj637 testdata.511299 2026-02-19T13:18:35.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj638 testdata.511299 2026-02-19T13:18:35.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj639 testdata.511299 2026-02-19T13:18:35.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj640 testdata.511299 2026-02-19T13:18:35.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj641 testdata.511299 2026-02-19T13:18:35.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj642 testdata.511299 2026-02-19T13:18:35.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj643 testdata.511299 2026-02-19T13:18:35.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj644 testdata.511299 2026-02-19T13:18:35.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj645 testdata.511299 2026-02-19T13:18:35.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj646 testdata.511299 2026-02-19T13:18:35.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj647 testdata.511299 2026-02-19T13:18:35.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj648 testdata.511299 2026-02-19T13:18:35.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj649 testdata.511299 2026-02-19T13:18:35.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj650 testdata.511299 2026-02-19T13:18:35.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj651 testdata.511299 2026-02-19T13:18:35.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj652 testdata.511299 2026-02-19T13:18:35.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj653 testdata.511299 2026-02-19T13:18:35.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj654 testdata.511299 2026-02-19T13:18:35.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj655 testdata.511299 2026-02-19T13:18:35.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj656 testdata.511299 2026-02-19T13:18:35.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj657 testdata.511299 2026-02-19T13:18:35.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj658 testdata.511299 2026-02-19T13:18:35.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj659 testdata.511299 2026-02-19T13:18:35.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj660 testdata.511299 2026-02-19T13:18:35.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj661 testdata.511299 2026-02-19T13:18:35.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj662 testdata.511299 2026-02-19T13:18:35.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj663 testdata.511299 2026-02-19T13:18:35.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj664 testdata.511299 2026-02-19T13:18:35.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj665 testdata.511299 2026-02-19T13:18:35.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj666 testdata.511299 2026-02-19T13:18:35.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj667 testdata.511299 2026-02-19T13:18:35.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj668 testdata.511299 2026-02-19T13:18:35.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj669 testdata.511299 2026-02-19T13:18:35.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:35.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj670 testdata.511299 2026-02-19T13:18:36.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj671 testdata.511299 2026-02-19T13:18:36.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj672 testdata.511299 2026-02-19T13:18:36.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj673 testdata.511299 2026-02-19T13:18:36.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj674 testdata.511299 2026-02-19T13:18:36.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj675 testdata.511299 2026-02-19T13:18:36.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj676 testdata.511299 2026-02-19T13:18:36.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj677 testdata.511299 2026-02-19T13:18:36.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj678 testdata.511299 2026-02-19T13:18:36.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj679 testdata.511299 2026-02-19T13:18:36.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj680 testdata.511299 2026-02-19T13:18:36.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj681 testdata.511299 2026-02-19T13:18:36.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj682 testdata.511299 2026-02-19T13:18:36.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj683 testdata.511299 2026-02-19T13:18:36.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj684 testdata.511299 2026-02-19T13:18:36.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj685 testdata.511299 2026-02-19T13:18:36.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj686 testdata.511299 2026-02-19T13:18:36.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj687 testdata.511299 2026-02-19T13:18:36.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj688 testdata.511299 2026-02-19T13:18:36.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj689 testdata.511299 2026-02-19T13:18:36.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj690 testdata.511299 2026-02-19T13:18:36.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj691 testdata.511299 2026-02-19T13:18:36.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj692 testdata.511299 2026-02-19T13:18:36.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj693 testdata.511299 2026-02-19T13:18:36.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj694 testdata.511299 2026-02-19T13:18:36.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj695 testdata.511299 2026-02-19T13:18:36.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj696 testdata.511299 2026-02-19T13:18:36.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj697 testdata.511299 2026-02-19T13:18:36.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj698 testdata.511299 2026-02-19T13:18:36.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj699 testdata.511299 2026-02-19T13:18:36.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj700 testdata.511299 2026-02-19T13:18:36.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj701 testdata.511299 2026-02-19T13:18:36.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj702 testdata.511299 2026-02-19T13:18:36.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj703 testdata.511299 2026-02-19T13:18:36.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj704 testdata.511299 2026-02-19T13:18:36.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj705 testdata.511299 2026-02-19T13:18:36.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj706 testdata.511299 2026-02-19T13:18:36.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj707 testdata.511299 2026-02-19T13:18:36.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj708 testdata.511299 2026-02-19T13:18:36.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj709 testdata.511299 2026-02-19T13:18:36.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj710 testdata.511299 2026-02-19T13:18:36.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj711 testdata.511299 2026-02-19T13:18:36.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:36.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj712 testdata.511299 2026-02-19T13:18:37.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj713 testdata.511299 2026-02-19T13:18:37.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj714 testdata.511299 2026-02-19T13:18:37.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj715 testdata.511299 2026-02-19T13:18:37.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj716 testdata.511299 2026-02-19T13:18:37.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj717 testdata.511299 2026-02-19T13:18:37.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj718 testdata.511299 2026-02-19T13:18:37.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj719 testdata.511299 2026-02-19T13:18:37.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj720 testdata.511299 2026-02-19T13:18:37.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj721 testdata.511299 2026-02-19T13:18:37.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj722 testdata.511299 2026-02-19T13:18:37.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj723 testdata.511299 2026-02-19T13:18:37.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj724 testdata.511299 2026-02-19T13:18:37.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj725 testdata.511299 2026-02-19T13:18:37.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj726 testdata.511299 2026-02-19T13:18:37.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj727 testdata.511299 2026-02-19T13:18:37.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj728 testdata.511299 2026-02-19T13:18:37.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj729 testdata.511299 2026-02-19T13:18:37.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj730 testdata.511299 2026-02-19T13:18:37.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj731 testdata.511299 2026-02-19T13:18:37.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj732 testdata.511299 2026-02-19T13:18:37.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj733 testdata.511299 2026-02-19T13:18:37.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj734 testdata.511299 2026-02-19T13:18:37.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj735 testdata.511299 2026-02-19T13:18:37.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj736 testdata.511299 2026-02-19T13:18:37.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj737 testdata.511299 2026-02-19T13:18:37.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj738 testdata.511299 2026-02-19T13:18:37.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj739 testdata.511299 2026-02-19T13:18:37.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj740 testdata.511299 2026-02-19T13:18:37.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj741 testdata.511299 2026-02-19T13:18:37.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj742 testdata.511299 2026-02-19T13:18:37.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj743 testdata.511299 2026-02-19T13:18:37.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj744 testdata.511299 2026-02-19T13:18:37.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj745 testdata.511299 2026-02-19T13:18:37.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj746 testdata.511299 2026-02-19T13:18:37.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj747 testdata.511299 2026-02-19T13:18:37.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj748 testdata.511299 2026-02-19T13:18:37.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj749 testdata.511299 2026-02-19T13:18:37.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj750 testdata.511299 2026-02-19T13:18:37.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj751 testdata.511299 2026-02-19T13:18:37.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj752 testdata.511299 2026-02-19T13:18:37.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj753 testdata.511299 2026-02-19T13:18:37.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:37.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj754 testdata.511299 2026-02-19T13:18:38.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj755 testdata.511299 2026-02-19T13:18:38.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj756 testdata.511299 2026-02-19T13:18:38.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj757 testdata.511299 2026-02-19T13:18:38.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj758 testdata.511299 2026-02-19T13:18:38.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj759 testdata.511299 2026-02-19T13:18:38.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj760 testdata.511299 2026-02-19T13:18:38.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj761 testdata.511299 2026-02-19T13:18:38.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj762 testdata.511299 2026-02-19T13:18:38.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj763 testdata.511299 2026-02-19T13:18:38.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj764 testdata.511299 2026-02-19T13:18:38.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj765 testdata.511299 2026-02-19T13:18:38.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj766 testdata.511299 2026-02-19T13:18:38.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj767 testdata.511299 2026-02-19T13:18:38.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj768 testdata.511299 2026-02-19T13:18:38.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj769 testdata.511299 2026-02-19T13:18:38.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj770 testdata.511299 2026-02-19T13:18:38.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj771 testdata.511299 2026-02-19T13:18:38.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj772 testdata.511299 2026-02-19T13:18:38.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj773 testdata.511299 2026-02-19T13:18:38.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj774 testdata.511299 2026-02-19T13:18:38.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj775 testdata.511299 2026-02-19T13:18:38.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj776 testdata.511299 2026-02-19T13:18:38.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj777 testdata.511299 2026-02-19T13:18:38.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj778 testdata.511299 2026-02-19T13:18:38.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj779 testdata.511299 2026-02-19T13:18:38.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj780 testdata.511299 2026-02-19T13:18:38.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj781 testdata.511299 2026-02-19T13:18:38.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj782 testdata.511299 2026-02-19T13:18:38.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj783 testdata.511299 2026-02-19T13:18:38.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj784 testdata.511299 2026-02-19T13:18:38.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj785 testdata.511299 2026-02-19T13:18:38.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj786 testdata.511299 2026-02-19T13:18:38.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj787 testdata.511299 2026-02-19T13:18:38.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj788 testdata.511299 2026-02-19T13:18:38.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj789 testdata.511299 2026-02-19T13:18:38.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj790 testdata.511299 2026-02-19T13:18:38.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj791 testdata.511299 2026-02-19T13:18:38.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj792 testdata.511299 2026-02-19T13:18:38.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj793 testdata.511299 2026-02-19T13:18:38.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj794 testdata.511299 2026-02-19T13:18:38.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj795 testdata.511299 2026-02-19T13:18:38.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj796 testdata.511299 2026-02-19T13:18:38.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:38.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj797 testdata.511299 2026-02-19T13:18:39.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj798 testdata.511299 2026-02-19T13:18:39.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj799 testdata.511299 2026-02-19T13:18:39.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj800 testdata.511299 2026-02-19T13:18:39.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj801 testdata.511299 2026-02-19T13:18:39.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj802 testdata.511299 2026-02-19T13:18:39.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj803 testdata.511299 2026-02-19T13:18:39.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj804 testdata.511299 2026-02-19T13:18:39.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj805 testdata.511299 2026-02-19T13:18:39.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj806 testdata.511299 2026-02-19T13:18:39.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj807 testdata.511299 2026-02-19T13:18:39.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj808 testdata.511299 2026-02-19T13:18:39.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj809 testdata.511299 2026-02-19T13:18:39.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj810 testdata.511299 2026-02-19T13:18:39.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj811 testdata.511299 2026-02-19T13:18:39.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj812 testdata.511299 2026-02-19T13:18:39.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj813 testdata.511299 2026-02-19T13:18:39.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj814 testdata.511299 2026-02-19T13:18:39.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj815 testdata.511299 2026-02-19T13:18:39.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj816 testdata.511299 2026-02-19T13:18:39.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj817 testdata.511299 2026-02-19T13:18:39.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj818 testdata.511299 2026-02-19T13:18:39.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj819 testdata.511299 2026-02-19T13:18:39.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj820 testdata.511299 2026-02-19T13:18:39.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj821 testdata.511299 2026-02-19T13:18:39.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj822 testdata.511299 2026-02-19T13:18:39.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj823 testdata.511299 2026-02-19T13:18:39.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj824 testdata.511299 2026-02-19T13:18:39.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj825 testdata.511299 2026-02-19T13:18:39.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj826 testdata.511299 2026-02-19T13:18:39.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj827 testdata.511299 2026-02-19T13:18:39.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj828 testdata.511299 2026-02-19T13:18:39.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj829 testdata.511299 2026-02-19T13:18:39.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj830 testdata.511299 2026-02-19T13:18:39.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj831 testdata.511299 2026-02-19T13:18:39.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj832 testdata.511299 2026-02-19T13:18:39.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj833 testdata.511299 2026-02-19T13:18:39.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj834 testdata.511299 2026-02-19T13:18:39.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj835 testdata.511299 2026-02-19T13:18:39.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj836 testdata.511299 2026-02-19T13:18:39.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj837 testdata.511299 2026-02-19T13:18:39.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj838 testdata.511299 2026-02-19T13:18:39.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj839 testdata.511299 2026-02-19T13:18:39.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:39.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj840 testdata.511299 2026-02-19T13:18:40.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj841 testdata.511299 2026-02-19T13:18:40.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj842 testdata.511299 2026-02-19T13:18:40.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj843 testdata.511299 2026-02-19T13:18:40.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj844 testdata.511299 2026-02-19T13:18:40.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj845 testdata.511299 2026-02-19T13:18:40.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj846 testdata.511299 2026-02-19T13:18:40.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj847 testdata.511299 2026-02-19T13:18:40.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj848 testdata.511299 2026-02-19T13:18:40.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj849 testdata.511299 2026-02-19T13:18:40.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj850 testdata.511299 2026-02-19T13:18:40.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj851 testdata.511299 2026-02-19T13:18:40.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj852 testdata.511299 2026-02-19T13:18:40.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj853 testdata.511299 2026-02-19T13:18:40.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj854 testdata.511299 2026-02-19T13:18:40.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj855 testdata.511299 2026-02-19T13:18:40.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj856 testdata.511299 2026-02-19T13:18:40.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj857 testdata.511299 2026-02-19T13:18:40.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj858 testdata.511299 2026-02-19T13:18:40.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj859 testdata.511299 2026-02-19T13:18:40.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj860 testdata.511299 2026-02-19T13:18:40.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj861 testdata.511299 2026-02-19T13:18:40.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj862 testdata.511299 2026-02-19T13:18:40.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj863 testdata.511299 2026-02-19T13:18:40.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj864 testdata.511299 2026-02-19T13:18:40.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj865 testdata.511299 2026-02-19T13:18:40.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj866 testdata.511299 2026-02-19T13:18:40.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj867 testdata.511299 2026-02-19T13:18:40.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj868 testdata.511299 2026-02-19T13:18:40.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj869 testdata.511299 2026-02-19T13:18:40.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj870 testdata.511299 2026-02-19T13:18:40.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj871 testdata.511299 2026-02-19T13:18:40.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj872 testdata.511299 2026-02-19T13:18:40.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj873 testdata.511299 2026-02-19T13:18:40.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj874 testdata.511299 2026-02-19T13:18:40.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj875 testdata.511299 2026-02-19T13:18:40.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj876 testdata.511299 2026-02-19T13:18:40.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj877 testdata.511299 2026-02-19T13:18:40.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj878 testdata.511299 2026-02-19T13:18:40.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj879 testdata.511299 2026-02-19T13:18:40.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj880 testdata.511299 2026-02-19T13:18:40.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj881 testdata.511299 2026-02-19T13:18:40.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:40.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj882 testdata.511299 2026-02-19T13:18:41.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj883 testdata.511299 2026-02-19T13:18:41.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj884 testdata.511299 2026-02-19T13:18:41.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj885 testdata.511299 2026-02-19T13:18:41.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj886 testdata.511299 2026-02-19T13:18:41.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj887 testdata.511299 2026-02-19T13:18:41.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj888 testdata.511299 2026-02-19T13:18:41.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj889 testdata.511299 2026-02-19T13:18:41.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj890 testdata.511299 2026-02-19T13:18:41.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj891 testdata.511299 2026-02-19T13:18:41.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj892 testdata.511299 2026-02-19T13:18:41.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj893 testdata.511299 2026-02-19T13:18:41.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj894 testdata.511299 2026-02-19T13:18:41.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj895 testdata.511299 2026-02-19T13:18:41.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj896 testdata.511299 2026-02-19T13:18:41.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj897 testdata.511299 2026-02-19T13:18:41.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj898 testdata.511299 2026-02-19T13:18:41.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj899 testdata.511299 2026-02-19T13:18:41.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj900 testdata.511299 2026-02-19T13:18:41.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj901 testdata.511299 2026-02-19T13:18:41.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj902 testdata.511299 2026-02-19T13:18:41.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj903 testdata.511299 2026-02-19T13:18:41.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj904 testdata.511299 2026-02-19T13:18:41.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj905 testdata.511299 2026-02-19T13:18:41.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj906 testdata.511299 2026-02-19T13:18:41.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj907 testdata.511299 2026-02-19T13:18:41.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj908 testdata.511299 2026-02-19T13:18:41.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj909 testdata.511299 2026-02-19T13:18:41.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj910 testdata.511299 2026-02-19T13:18:41.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj911 testdata.511299 2026-02-19T13:18:41.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj912 testdata.511299 2026-02-19T13:18:41.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj913 testdata.511299 2026-02-19T13:18:41.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj914 testdata.511299 2026-02-19T13:18:41.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj915 testdata.511299 2026-02-19T13:18:41.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj916 testdata.511299 2026-02-19T13:18:41.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj917 testdata.511299 2026-02-19T13:18:41.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj918 testdata.511299 2026-02-19T13:18:41.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj919 testdata.511299 2026-02-19T13:18:41.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj920 testdata.511299 2026-02-19T13:18:41.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj921 testdata.511299 2026-02-19T13:18:41.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj922 testdata.511299 2026-02-19T13:18:41.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj923 testdata.511299 2026-02-19T13:18:41.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:41.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj924 testdata.511299 2026-02-19T13:18:42.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj925 testdata.511299 2026-02-19T13:18:42.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj926 testdata.511299 2026-02-19T13:18:42.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj927 testdata.511299 2026-02-19T13:18:42.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj928 testdata.511299 2026-02-19T13:18:42.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj929 testdata.511299 2026-02-19T13:18:42.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj930 testdata.511299 2026-02-19T13:18:42.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj931 testdata.511299 2026-02-19T13:18:42.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj932 testdata.511299 2026-02-19T13:18:42.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj933 testdata.511299 2026-02-19T13:18:42.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj934 testdata.511299 2026-02-19T13:18:42.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj935 testdata.511299 2026-02-19T13:18:42.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj936 testdata.511299 2026-02-19T13:18:42.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj937 testdata.511299 2026-02-19T13:18:42.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj938 testdata.511299 2026-02-19T13:18:42.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj939 testdata.511299 2026-02-19T13:18:42.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj940 testdata.511299 2026-02-19T13:18:42.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj941 testdata.511299 2026-02-19T13:18:42.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj942 testdata.511299 2026-02-19T13:18:42.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj943 testdata.511299 2026-02-19T13:18:42.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj944 testdata.511299 2026-02-19T13:18:42.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj945 testdata.511299 2026-02-19T13:18:42.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj946 testdata.511299 2026-02-19T13:18:42.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj947 testdata.511299 2026-02-19T13:18:42.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj948 testdata.511299 2026-02-19T13:18:42.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj949 testdata.511299 2026-02-19T13:18:42.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj950 testdata.511299 2026-02-19T13:18:42.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj951 testdata.511299 2026-02-19T13:18:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj952 testdata.511299 2026-02-19T13:18:42.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj953 testdata.511299 2026-02-19T13:18:42.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj954 testdata.511299 2026-02-19T13:18:42.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj955 testdata.511299 2026-02-19T13:18:42.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj956 testdata.511299 2026-02-19T13:18:42.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj957 testdata.511299 2026-02-19T13:18:42.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj958 testdata.511299 2026-02-19T13:18:42.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj959 testdata.511299 2026-02-19T13:18:42.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj960 testdata.511299 2026-02-19T13:18:42.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj961 testdata.511299 2026-02-19T13:18:42.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj962 testdata.511299 2026-02-19T13:18:42.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj963 testdata.511299 2026-02-19T13:18:42.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj964 testdata.511299 2026-02-19T13:18:42.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:42.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj965 testdata.511299 2026-02-19T13:18:43.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj966 testdata.511299 2026-02-19T13:18:43.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj967 testdata.511299 2026-02-19T13:18:43.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj968 testdata.511299 2026-02-19T13:18:43.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj969 testdata.511299 2026-02-19T13:18:43.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj970 testdata.511299 2026-02-19T13:18:43.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj971 testdata.511299 2026-02-19T13:18:43.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj972 testdata.511299 2026-02-19T13:18:43.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj973 testdata.511299 2026-02-19T13:18:43.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj974 testdata.511299 2026-02-19T13:18:43.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj975 testdata.511299 2026-02-19T13:18:43.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj976 testdata.511299 2026-02-19T13:18:43.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj977 testdata.511299 2026-02-19T13:18:43.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj978 testdata.511299 2026-02-19T13:18:43.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj979 testdata.511299 2026-02-19T13:18:43.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj980 testdata.511299 2026-02-19T13:18:43.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj981 testdata.511299 2026-02-19T13:18:43.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj982 testdata.511299 2026-02-19T13:18:43.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj983 testdata.511299 2026-02-19T13:18:43.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj984 testdata.511299 2026-02-19T13:18:43.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj985 testdata.511299 2026-02-19T13:18:43.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj986 testdata.511299 2026-02-19T13:18:43.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj987 testdata.511299 2026-02-19T13:18:43.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj988 testdata.511299 2026-02-19T13:18:43.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj989 testdata.511299 2026-02-19T13:18:43.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj990 testdata.511299 2026-02-19T13:18:43.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj991 testdata.511299 2026-02-19T13:18:43.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj992 testdata.511299 2026-02-19T13:18:43.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj993 testdata.511299 2026-02-19T13:18:43.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj994 testdata.511299 2026-02-19T13:18:43.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj995 testdata.511299 2026-02-19T13:18:43.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj996 testdata.511299 2026-02-19T13:18:43.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj997 testdata.511299 2026-02-19T13:18:43.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj998 testdata.511299 2026-02-19T13:18:43.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj999 testdata.511299 2026-02-19T13:18:43.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:18:43.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj1000 testdata.511299 2026-02-19T13:18:43.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:334: _scrub_abort: rm -f testdata.511299 2026-02-19T13:18:43.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:336: _scrub_abort: get_primary test obj1 2026-02-19T13:18:43.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=test 2026-02-19T13:18:43.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T13:18:43.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map test obj1 2026-02-19T13:18:43.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:18:44.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:336: _scrub_abort: local primary=1 2026-02-19T13:18:44.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: local pgid=1.0 2026-02-19T13:18:44.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:339: _scrub_abort: ceph tell 1.0 schedule-deep-scrub 2026-02-19T13:18:44.132 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:18:44.132 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T13:18:44.132 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:18:44.132 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-08T08:30:34.133441+0000" 2026-02-19T13:18:44.132 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:18:44.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:342: _scrub_abort: set -o pipefail 2026-02-19T13:18:44.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:343: _scrub_abort: found=no 2026-02-19T13:18:44.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:344: _scrub_abort: seq 0 200 2026-02-19T13:18:44.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:344: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:18:44.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:346: _scrub_abort: flush_pg_stats 2026-02-19T13:18:44.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:44.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:44.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:44.304 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:18:44.304 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:18:44.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:44.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:44.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:44.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836515 2026-02-19T13:18:44.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836515 2026-02-19T13:18:44.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836515' 2026-02-19T13:18:44.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:44.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:44.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705696 2026-02-19T13:18:44.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705696 2026-02-19T13:18:44.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836515 1-38654705696' 2026-02-19T13:18:44.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:44.447 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:44.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574878 2026-02-19T13:18:44.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574878 2026-02-19T13:18:44.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836515 1-38654705696 2-55834574878' 2026-02-19T13:18:44.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:44.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836515 2026-02-19T13:18:44.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:44.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:44.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836515 2026-02-19T13:18:44.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:44.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836515 2026-02-19T13:18:44.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836515' 2026-02-19T13:18:44.525 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836515 2026-02-19T13:18:44.525 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:44.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836513 -lt 21474836515 2026-02-19T13:18:44.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:18:45.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:18:45.682 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:45.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836516 -lt 21474836515 2026-02-19T13:18:45.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:45.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705696 2026-02-19T13:18:45.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:45.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:45.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705696 2026-02-19T13:18:45.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:45.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705696 2026-02-19T13:18:45.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705696' 2026-02-19T13:18:45.847 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705696 2026-02-19T13:18:45.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:46.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705697 -lt 38654705696 2026-02-19T13:18:46.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:46.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574878 2026-02-19T13:18:46.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:46.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:46.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574878 2026-02-19T13:18:46.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:46.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574878 2026-02-19T13:18:46.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574878' 2026-02-19T13:18:46.014 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574878 2026-02-19T13:18:46.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:46.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574879 -lt 55834574878 2026-02-19T13:18:46.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: ceph pg dump pgs 2026-02-19T13:18:46.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: grep '^1.0' 2026-02-19T13:18:46.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: grep -q scrubbing 2026-02-19T13:18:46.314 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:18:46.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:344: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:18:46.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:346: _scrub_abort: flush_pg_stats 2026-02-19T13:18:46.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:46.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:46.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:46.484 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:18:46.484 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:18:46.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:46.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:46.485 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:46.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836518 2026-02-19T13:18:46.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836518 2026-02-19T13:18:46.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518' 2026-02-19T13:18:46.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:46.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:46.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705700 2026-02-19T13:18:46.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705700 2026-02-19T13:18:46.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518 1-38654705700' 2026-02-19T13:18:46.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:46.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:46.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574881 2026-02-19T13:18:46.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574881 2026-02-19T13:18:46.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518 1-38654705700 2-55834574881' 2026-02-19T13:18:46.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:46.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836518 2026-02-19T13:18:46.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:46.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:46.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836518 2026-02-19T13:18:46.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:46.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836518 2026-02-19T13:18:46.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836518' 2026-02-19T13:18:46.709 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836518 2026-02-19T13:18:46.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:46.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836516 -lt 21474836518 2026-02-19T13:18:46.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:18:47.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:18:47.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:48.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836519 -lt 21474836518 2026-02-19T13:18:48.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:48.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705700 2026-02-19T13:18:48.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:48.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:48.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705700 2026-02-19T13:18:48.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:48.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705700 2026-02-19T13:18:48.039 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705700 2026-02-19T13:18:48.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705700' 2026-02-19T13:18:48.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:48.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705700 -lt 38654705700 2026-02-19T13:18:48.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:48.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574881 2026-02-19T13:18:48.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:48.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:48.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574881 2026-02-19T13:18:48.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:48.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574881 2026-02-19T13:18:48.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574881' 2026-02-19T13:18:48.200 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574881 2026-02-19T13:18:48.200 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:48.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574882 -lt 55834574881 2026-02-19T13:18:48.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: ceph pg dump pgs 2026-02-19T13:18:48.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: grep '^1.0' 2026-02-19T13:18:48.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: grep -q scrubbing 2026-02-19T13:18:48.507 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:18:48.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:349: _scrub_abort: found=yes 2026-02-19T13:18:48.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:351: _scrub_abort: break 2026-02-19T13:18:48.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:354: _scrub_abort: set +o pipefail 2026-02-19T13:18:48.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:356: _scrub_abort: test yes = no 2026-02-19T13:18:48.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:362: _scrub_abort: ceph osd set nodeep-scrub 2026-02-19T13:18:48.717 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:18:48.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:363: _scrub_abort: '[' deep-scrub = deep-scrub ']' 2026-02-19T13:18:48.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:365: _scrub_abort: ceph osd set noscrub 2026-02-19T13:18:48.922 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:18:48.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:369: _scrub_abort: set -o pipefail 2026-02-19T13:18:48.934 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: seq 0 200 2026-02-19T13:18:48.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:18:48.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:18:48.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:48.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:49.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:49.092 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:18:49.092 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:18:49.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:49.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:49.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:49.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836522 2026-02-19T13:18:49.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836522 2026-02-19T13:18:49.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836522' 2026-02-19T13:18:49.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:49.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:49.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705703 2026-02-19T13:18:49.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705703 2026-02-19T13:18:49.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836522 1-38654705703' 2026-02-19T13:18:49.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:49.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:49.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574885 2026-02-19T13:18:49.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574885 2026-02-19T13:18:49.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836522 1-38654705703 2-55834574885' 2026-02-19T13:18:49.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:49.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836522 2026-02-19T13:18:49.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:49.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:49.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836522 2026-02-19T13:18:49.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:49.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836522 2026-02-19T13:18:49.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836522' 2026-02-19T13:18:49.318 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836522 2026-02-19T13:18:49.318 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:49.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836522 -lt 21474836522 2026-02-19T13:18:49.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:49.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705703 2026-02-19T13:18:49.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:49.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:49.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705703 2026-02-19T13:18:49.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:49.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705703 2026-02-19T13:18:49.486 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705703 2026-02-19T13:18:49.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705703' 2026-02-19T13:18:49.487 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:49.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705703 -lt 38654705703 2026-02-19T13:18:49.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:49.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574885 2026-02-19T13:18:49.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:49.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:49.657 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574885 2026-02-19T13:18:49.657 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:49.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574885 2026-02-19T13:18:49.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574885' 2026-02-19T13:18:49.658 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574885 2026-02-19T13:18:49.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:49.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574884 -lt 55834574885 2026-02-19T13:18:49.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:18:50.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:18:50.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:50.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574884 -lt 55834574885 2026-02-19T13:18:50.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:18:51.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:18:51.989 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:52.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574887 -lt 55834574885 2026-02-19T13:18:52.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:18:52.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:18:52.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:18:52.296 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:18:52.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:18:52.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:18:52.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:18:52.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:52.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:52.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:52.462 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:18:52.462 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:18:52.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:52.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:52.463 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:52.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836526 2026-02-19T13:18:52.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836526 2026-02-19T13:18:52.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836526' 2026-02-19T13:18:52.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:52.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:52.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705708 2026-02-19T13:18:52.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705708 2026-02-19T13:18:52.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836526 1-38654705708' 2026-02-19T13:18:52.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:52.606 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:52.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574889 2026-02-19T13:18:52.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574889 2026-02-19T13:18:52.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836526 1-38654705708 2-55834574889' 2026-02-19T13:18:52.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:52.678 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836526 2026-02-19T13:18:52.678 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:52.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:52.679 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836526 2026-02-19T13:18:52.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:52.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836526 2026-02-19T13:18:52.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836526' 2026-02-19T13:18:52.681 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836526 2026-02-19T13:18:52.682 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:52.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836524 -lt 21474836526 2026-02-19T13:18:52.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:18:53.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:18:53.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:54.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836527 -lt 21474836526 2026-02-19T13:18:54.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:54.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705708 2026-02-19T13:18:54.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:54.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:54.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705708 2026-02-19T13:18:54.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:54.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705708 2026-02-19T13:18:54.006 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705708 2026-02-19T13:18:54.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705708' 2026-02-19T13:18:54.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:54.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705708 -lt 38654705708 2026-02-19T13:18:54.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:54.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574889 2026-02-19T13:18:54.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:54.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:54.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574889 2026-02-19T13:18:54.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:54.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574889 2026-02-19T13:18:54.172 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574889 2026-02-19T13:18:54.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574889' 2026-02-19T13:18:54.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:54.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574890 -lt 55834574889 2026-02-19T13:18:54.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:18:54.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:18:54.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:18:54.473 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:18:54.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:18:54.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:18:54.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:18:54.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:54.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:54.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:54.639 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:18:54.639 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:18:54.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:54.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:54.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:54.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836529 2026-02-19T13:18:54.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836529 2026-02-19T13:18:54.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836529' 2026-02-19T13:18:54.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:54.714 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:54.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705711 2026-02-19T13:18:54.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705711 2026-02-19T13:18:54.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836529 1-38654705711' 2026-02-19T13:18:54.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:54.785 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:54.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574892 2026-02-19T13:18:54.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574892 2026-02-19T13:18:54.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836529 1-38654705711 2-55834574892' 2026-02-19T13:18:54.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:54.858 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836529 2026-02-19T13:18:54.858 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:54.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:54.858 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836529 2026-02-19T13:18:54.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:54.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836529 2026-02-19T13:18:54.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836529' 2026-02-19T13:18:54.860 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836529 2026-02-19T13:18:54.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:55.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836527 -lt 21474836529 2026-02-19T13:18:55.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:18:56.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:18:56.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:56.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836530 -lt 21474836529 2026-02-19T13:18:56.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:56.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705711 2026-02-19T13:18:56.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:56.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:56.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705711 2026-02-19T13:18:56.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:56.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705711 2026-02-19T13:18:56.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705711' 2026-02-19T13:18:56.177 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705711 2026-02-19T13:18:56.177 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:56.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705711 -lt 38654705711 2026-02-19T13:18:56.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:56.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574892 2026-02-19T13:18:56.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:56.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:56.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574892 2026-02-19T13:18:56.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:56.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574892 2026-02-19T13:18:56.348 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574892 2026-02-19T13:18:56.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574892' 2026-02-19T13:18:56.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:56.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574893 -lt 55834574892 2026-02-19T13:18:56.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:18:56.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:18:56.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:18:56.658 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:18:56.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:18:56.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:18:56.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:18:56.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:56.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:56.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:56.821 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:18:56.821 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:18:56.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:56.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:56.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:56.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836532 2026-02-19T13:18:56.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836532 2026-02-19T13:18:56.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836532' 2026-02-19T13:18:56.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:56.893 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:56.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705714 2026-02-19T13:18:56.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705714 2026-02-19T13:18:56.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836532 1-38654705714' 2026-02-19T13:18:56.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:56.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:57.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574896 2026-02-19T13:18:57.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574896 2026-02-19T13:18:57.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836532 1-38654705714 2-55834574896' 2026-02-19T13:18:57.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:57.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836532 2026-02-19T13:18:57.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:57.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:57.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836532 2026-02-19T13:18:57.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:57.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836532 2026-02-19T13:18:57.039 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836532 2026-02-19T13:18:57.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836532' 2026-02-19T13:18:57.039 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:57.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836530 -lt 21474836532 2026-02-19T13:18:57.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:18:58.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:18:58.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:58.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836533 -lt 21474836532 2026-02-19T13:18:58.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:58.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705714 2026-02-19T13:18:58.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:58.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:58.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705714 2026-02-19T13:18:58.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:58.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705714 2026-02-19T13:18:58.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705714' 2026-02-19T13:18:58.362 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705714 2026-02-19T13:18:58.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:58.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705714 -lt 38654705714 2026-02-19T13:18:58.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:58.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574896 2026-02-19T13:18:58.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:58.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:58.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574896 2026-02-19T13:18:58.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:58.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574896 2026-02-19T13:18:58.530 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574896 2026-02-19T13:18:58.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574896' 2026-02-19T13:18:58.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:58.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574896 -lt 55834574896 2026-02-19T13:18:58.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:18:58.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:18:58.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:18:58.845 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:18:58.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:18:58.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:18:58.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:18:58.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:58.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:59.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:59.007 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:18:59.008 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:18:59.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:59.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:59.008 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:59.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836535 2026-02-19T13:18:59.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836535 2026-02-19T13:18:59.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836535' 2026-02-19T13:18:59.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:59.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:59.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705717 2026-02-19T13:18:59.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705717 2026-02-19T13:18:59.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836535 1-38654705717' 2026-02-19T13:18:59.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:59.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:59.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574899 2026-02-19T13:18:59.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574899 2026-02-19T13:18:59.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836535 1-38654705717 2-55834574899' 2026-02-19T13:18:59.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:59.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836535 2026-02-19T13:18:59.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:59.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:59.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836535 2026-02-19T13:18:59.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:59.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836535 2026-02-19T13:18:59.229 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836535 2026-02-19T13:18:59.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836535' 2026-02-19T13:18:59.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:59.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836536 -lt 21474836535 2026-02-19T13:18:59.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:59.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705717 2026-02-19T13:18:59.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:59.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:59.383 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705717 2026-02-19T13:18:59.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:59.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705717 2026-02-19T13:18:59.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705717' 2026-02-19T13:18:59.385 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705717 2026-02-19T13:18:59.385 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:59.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705717 -lt 38654705717 2026-02-19T13:18:59.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:59.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574899 2026-02-19T13:18:59.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:59.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:59.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574899 2026-02-19T13:18:59.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:59.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574899 2026-02-19T13:18:59.561 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574899 2026-02-19T13:18:59.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574899' 2026-02-19T13:18:59.562 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:59.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574899 -lt 55834574899 2026-02-19T13:18:59.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:18:59.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:18:59.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:18:59.877 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:18:59.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:18:59.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:18:59.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:18:59.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:59.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:00.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:00.046 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:00.046 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:00.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:00.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:00.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:00.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836537 2026-02-19T13:19:00.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836537 2026-02-19T13:19:00.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836537' 2026-02-19T13:19:00.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:00.120 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:00.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705719 2026-02-19T13:19:00.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705719 2026-02-19T13:19:00.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836537 1-38654705719' 2026-02-19T13:19:00.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:00.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:00.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574901 2026-02-19T13:19:00.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574901 2026-02-19T13:19:00.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836537 1-38654705719 2-55834574901' 2026-02-19T13:19:00.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:00.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836537 2026-02-19T13:19:00.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:00.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:00.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836537 2026-02-19T13:19:00.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:00.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836537 2026-02-19T13:19:00.259 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836537 2026-02-19T13:19:00.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836537' 2026-02-19T13:19:00.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:00.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836536 -lt 21474836537 2026-02-19T13:19:00.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:01.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:01.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:01.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836539 -lt 21474836537 2026-02-19T13:19:01.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:01.574 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705719 2026-02-19T13:19:01.574 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:01.575 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:01.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705719 2026-02-19T13:19:01.576 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:01.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705719 2026-02-19T13:19:01.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705719' 2026-02-19T13:19:01.578 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705719 2026-02-19T13:19:01.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:01.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705720 -lt 38654705719 2026-02-19T13:19:01.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:01.744 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574901 2026-02-19T13:19:01.744 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:01.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:01.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574901 2026-02-19T13:19:01.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:01.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574901 2026-02-19T13:19:01.748 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574901 2026-02-19T13:19:01.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574901' 2026-02-19T13:19:01.748 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:01.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574902 -lt 55834574901 2026-02-19T13:19:01.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:01.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:01.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:02.047 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:02.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:02.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:02.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:02.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:02.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:02.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:02.212 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:02.212 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:02.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:02.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:02.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:02.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836541 2026-02-19T13:19:02.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836541 2026-02-19T13:19:02.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836541' 2026-02-19T13:19:02.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:02.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:02.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705722 2026-02-19T13:19:02.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705722 2026-02-19T13:19:02.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836541 1-38654705722' 2026-02-19T13:19:02.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:02.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:02.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574904 2026-02-19T13:19:02.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574904 2026-02-19T13:19:02.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836541 1-38654705722 2-55834574904' 2026-02-19T13:19:02.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:02.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836541 2026-02-19T13:19:02.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:02.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:02.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836541 2026-02-19T13:19:02.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:02.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836541 2026-02-19T13:19:02.434 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836541 2026-02-19T13:19:02.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836541' 2026-02-19T13:19:02.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:02.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836539 -lt 21474836541 2026-02-19T13:19:02.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:03.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:03.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:03.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836542 -lt 21474836541 2026-02-19T13:19:03.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:03.758 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705722 2026-02-19T13:19:03.759 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:03.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:03.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705722 2026-02-19T13:19:03.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:03.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705722 2026-02-19T13:19:03.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705722' 2026-02-19T13:19:03.762 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705722 2026-02-19T13:19:03.762 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:03.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705723 -lt 38654705722 2026-02-19T13:19:03.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:03.931 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574904 2026-02-19T13:19:03.931 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:03.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:03.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574904 2026-02-19T13:19:03.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:03.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574904 2026-02-19T13:19:03.935 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574904 2026-02-19T13:19:03.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574904' 2026-02-19T13:19:03.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:04.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574905 -lt 55834574904 2026-02-19T13:19:04.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:04.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:04.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:04.248 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:04.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:04.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:04.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:04.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:04.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:04.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:04.419 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:04.419 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:04.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:04.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:04.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:04.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836544 2026-02-19T13:19:04.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836544 2026-02-19T13:19:04.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836544' 2026-02-19T13:19:04.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:04.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:04.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705726 2026-02-19T13:19:04.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705726 2026-02-19T13:19:04.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836544 1-38654705726' 2026-02-19T13:19:04.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:04.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:04.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574907 2026-02-19T13:19:04.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574907 2026-02-19T13:19:04.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836544 1-38654705726 2-55834574907' 2026-02-19T13:19:04.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:04.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836544 2026-02-19T13:19:04.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:04.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:04.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836544 2026-02-19T13:19:04.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:04.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836544 2026-02-19T13:19:04.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836544' 2026-02-19T13:19:04.641 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836544 2026-02-19T13:19:04.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:04.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836542 -lt 21474836544 2026-02-19T13:19:04.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:05.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:05.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:05.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836545 -lt 21474836544 2026-02-19T13:19:05.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:05.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705726 2026-02-19T13:19:05.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:05.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:05.966 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705726 2026-02-19T13:19:05.966 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:05.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705726 2026-02-19T13:19:05.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705726' 2026-02-19T13:19:05.968 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705726 2026-02-19T13:19:05.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:06.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705726 -lt 38654705726 2026-02-19T13:19:06.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:06.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574907 2026-02-19T13:19:06.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:06.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:06.136 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574907 2026-02-19T13:19:06.136 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:06.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574907 2026-02-19T13:19:06.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574907' 2026-02-19T13:19:06.138 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574907 2026-02-19T13:19:06.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:06.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574908 -lt 55834574907 2026-02-19T13:19:06.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:06.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:06.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:06.450 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:06.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:06.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:06.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:06.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:06.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:06.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:06.620 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:06.620 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:06.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:06.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:06.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:06.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836547 2026-02-19T13:19:06.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836547 2026-02-19T13:19:06.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836547' 2026-02-19T13:19:06.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:06.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:06.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705729 2026-02-19T13:19:06.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705729 2026-02-19T13:19:06.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836547 1-38654705729' 2026-02-19T13:19:06.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:06.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:06.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574910 2026-02-19T13:19:06.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574910 2026-02-19T13:19:06.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836547 1-38654705729 2-55834574910' 2026-02-19T13:19:06.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:06.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836547 2026-02-19T13:19:06.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:06.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:06.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836547 2026-02-19T13:19:06.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:06.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836547 2026-02-19T13:19:06.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836547' 2026-02-19T13:19:06.842 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836547 2026-02-19T13:19:06.842 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:07.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836545 -lt 21474836547 2026-02-19T13:19:07.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:08.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:08.003 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:08.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836548 -lt 21474836547 2026-02-19T13:19:08.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:08.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705729 2026-02-19T13:19:08.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:08.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:08.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705729 2026-02-19T13:19:08.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:08.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705729 2026-02-19T13:19:08.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705729' 2026-02-19T13:19:08.167 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705729 2026-02-19T13:19:08.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:08.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705729 -lt 38654705729 2026-02-19T13:19:08.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:08.341 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574910 2026-02-19T13:19:08.341 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:08.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:08.343 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574910 2026-02-19T13:19:08.343 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:08.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574910 2026-02-19T13:19:08.344 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574910 2026-02-19T13:19:08.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574910' 2026-02-19T13:19:08.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:08.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574911 -lt 55834574910 2026-02-19T13:19:08.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:08.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:08.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:08.650 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:08.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:08.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:08.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:08.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:08.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:08.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:08.815 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:08.815 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:08.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:08.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:08.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:08.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836550 2026-02-19T13:19:08.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836550 2026-02-19T13:19:08.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836550' 2026-02-19T13:19:08.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:08.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:08.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705732 2026-02-19T13:19:08.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705732 2026-02-19T13:19:08.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836550 1-38654705732' 2026-02-19T13:19:08.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:08.961 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:09.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574914 2026-02-19T13:19:09.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574914 2026-02-19T13:19:09.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836550 1-38654705732 2-55834574914' 2026-02-19T13:19:09.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:09.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836550 2026-02-19T13:19:09.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:09.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:09.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836550 2026-02-19T13:19:09.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:09.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836550 2026-02-19T13:19:09.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836550' 2026-02-19T13:19:09.040 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836550 2026-02-19T13:19:09.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:09.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836548 -lt 21474836550 2026-02-19T13:19:09.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:10.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:10.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:10.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836551 -lt 21474836550 2026-02-19T13:19:10.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:10.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705732 2026-02-19T13:19:10.363 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:10.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:10.364 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705732 2026-02-19T13:19:10.364 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:10.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705732 2026-02-19T13:19:10.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705732' 2026-02-19T13:19:10.366 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705732 2026-02-19T13:19:10.366 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:10.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705732 -lt 38654705732 2026-02-19T13:19:10.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:10.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574914 2026-02-19T13:19:10.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:10.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:10.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574914 2026-02-19T13:19:10.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:10.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574914 2026-02-19T13:19:10.536 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574914 2026-02-19T13:19:10.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574914' 2026-02-19T13:19:10.537 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:10.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574914 -lt 55834574914 2026-02-19T13:19:10.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:10.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:10.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:10.844 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:10.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:10.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:10.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:10.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:10.856 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:11.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:11.012 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:11.013 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:11.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:11.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:11.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:11.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836553 2026-02-19T13:19:11.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836553 2026-02-19T13:19:11.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836553' 2026-02-19T13:19:11.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:11.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:11.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705735 2026-02-19T13:19:11.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705735 2026-02-19T13:19:11.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836553 1-38654705735' 2026-02-19T13:19:11.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:11.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:11.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574917 2026-02-19T13:19:11.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574917 2026-02-19T13:19:11.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836553 1-38654705735 2-55834574917' 2026-02-19T13:19:11.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:11.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836553 2026-02-19T13:19:11.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:11.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:11.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836553 2026-02-19T13:19:11.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:11.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836553 2026-02-19T13:19:11.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836553' 2026-02-19T13:19:11.234 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836553 2026-02-19T13:19:11.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:11.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836554 -lt 21474836553 2026-02-19T13:19:11.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:11.397 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705735 2026-02-19T13:19:11.397 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:11.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:11.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705735 2026-02-19T13:19:11.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:11.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705735 2026-02-19T13:19:11.400 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705735 2026-02-19T13:19:11.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705735' 2026-02-19T13:19:11.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:11.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705735 -lt 38654705735 2026-02-19T13:19:11.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:11.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574917 2026-02-19T13:19:11.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:11.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:11.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574917 2026-02-19T13:19:11.561 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:11.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574917 2026-02-19T13:19:11.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574917' 2026-02-19T13:19:11.562 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574917 2026-02-19T13:19:11.562 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:11.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574917 -lt 55834574917 2026-02-19T13:19:11.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:11.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:11.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:11.873 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:11.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:11.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:11.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:11.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:11.885 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:12.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:12.048 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:12.048 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:12.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:12.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:12.048 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836555 2026-02-19T13:19:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836555 2026-02-19T13:19:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836555' 2026-02-19T13:19:12.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:12.121 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:12.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705737 2026-02-19T13:19:12.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705737 2026-02-19T13:19:12.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836555 1-38654705737' 2026-02-19T13:19:12.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:12.194 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:12.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574919 2026-02-19T13:19:12.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574919 2026-02-19T13:19:12.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836555 1-38654705737 2-55834574919' 2026-02-19T13:19:12.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:12.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836555 2026-02-19T13:19:12.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:12.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:12.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836555 2026-02-19T13:19:12.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:12.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836555 2026-02-19T13:19:12.266 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836555 2026-02-19T13:19:12.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836555' 2026-02-19T13:19:12.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:12.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836554 -lt 21474836555 2026-02-19T13:19:12.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:13.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:13.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:13.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836557 -lt 21474836555 2026-02-19T13:19:13.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:13.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705737 2026-02-19T13:19:13.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:13.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:13.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705737 2026-02-19T13:19:13.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:13.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705737 2026-02-19T13:19:13.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705737' 2026-02-19T13:19:13.597 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705737 2026-02-19T13:19:13.597 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:13.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705738 -lt 38654705737 2026-02-19T13:19:13.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:13.768 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574919 2026-02-19T13:19:13.768 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:13.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:13.770 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574919 2026-02-19T13:19:13.770 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:13.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574919 2026-02-19T13:19:13.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574919' 2026-02-19T13:19:13.772 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574919 2026-02-19T13:19:13.772 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:13.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574920 -lt 55834574919 2026-02-19T13:19:13.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:13.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:13.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:14.078 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:14.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:14.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:14.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:14.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:14.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:14.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:14.239 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:14.240 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:14.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:14.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:14.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:14.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836559 2026-02-19T13:19:14.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836559 2026-02-19T13:19:14.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836559' 2026-02-19T13:19:14.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:14.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:14.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705740 2026-02-19T13:19:14.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705740 2026-02-19T13:19:14.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836559 1-38654705740' 2026-02-19T13:19:14.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:14.387 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:14.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574922 2026-02-19T13:19:14.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574922 2026-02-19T13:19:14.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836559 1-38654705740 2-55834574922' 2026-02-19T13:19:14.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:14.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836559 2026-02-19T13:19:14.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:14.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:14.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836559 2026-02-19T13:19:14.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:14.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836559 2026-02-19T13:19:14.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836559' 2026-02-19T13:19:14.462 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836559 2026-02-19T13:19:14.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:14.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836557 -lt 21474836559 2026-02-19T13:19:14.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:15.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:15.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:15.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836560 -lt 21474836559 2026-02-19T13:19:15.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:15.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705740 2026-02-19T13:19:15.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:15.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:15.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705740 2026-02-19T13:19:15.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:15.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705740 2026-02-19T13:19:15.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705740' 2026-02-19T13:19:15.786 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705740 2026-02-19T13:19:15.786 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:15.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705741 -lt 38654705740 2026-02-19T13:19:15.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:15.952 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574922 2026-02-19T13:19:15.953 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:15.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:15.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574922 2026-02-19T13:19:15.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:15.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574922 2026-02-19T13:19:15.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574922' 2026-02-19T13:19:15.956 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574922 2026-02-19T13:19:15.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:16.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574923 -lt 55834574922 2026-02-19T13:19:16.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:16.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:16.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:16.266 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:16.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:16.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:16.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:16.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:16.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:16.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:16.434 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:16.435 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:16.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:16.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:16.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:16.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836562 2026-02-19T13:19:16.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836562 2026-02-19T13:19:16.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836562' 2026-02-19T13:19:16.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:16.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:16.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705744 2026-02-19T13:19:16.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705744 2026-02-19T13:19:16.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836562 1-38654705744' 2026-02-19T13:19:16.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:16.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:16.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574925 2026-02-19T13:19:16.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574925 2026-02-19T13:19:16.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836562 1-38654705744 2-55834574925' 2026-02-19T13:19:16.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:16.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836562 2026-02-19T13:19:16.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:16.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:16.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836562 2026-02-19T13:19:16.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:16.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836562 2026-02-19T13:19:16.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836562' 2026-02-19T13:19:16.654 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836562 2026-02-19T13:19:16.654 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:16.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836560 -lt 21474836562 2026-02-19T13:19:16.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:17.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:17.819 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:17.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836563 -lt 21474836562 2026-02-19T13:19:17.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:17.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705744 2026-02-19T13:19:17.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:17.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:17.978 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705744 2026-02-19T13:19:17.978 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:17.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705744 2026-02-19T13:19:17.980 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705744 2026-02-19T13:19:17.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705744' 2026-02-19T13:19:17.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:18.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705744 -lt 38654705744 2026-02-19T13:19:18.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:18.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574925 2026-02-19T13:19:18.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:18.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:18.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574925 2026-02-19T13:19:18.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:18.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574925 2026-02-19T13:19:18.145 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574925 2026-02-19T13:19:18.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574925' 2026-02-19T13:19:18.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:18.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574926 -lt 55834574925 2026-02-19T13:19:18.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:18.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:18.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:18.444 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:18.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:18.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:18.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:18.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:18.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:18.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:18.617 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:18.617 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:18.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:18.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:18.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:18.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836565 2026-02-19T13:19:18.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836565 2026-02-19T13:19:18.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836565' 2026-02-19T13:19:18.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:18.691 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:18.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705747 2026-02-19T13:19:18.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705747 2026-02-19T13:19:18.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836565 1-38654705747' 2026-02-19T13:19:18.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:18.762 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:18.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574928 2026-02-19T13:19:18.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574928 2026-02-19T13:19:18.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836565 1-38654705747 2-55834574928' 2026-02-19T13:19:18.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:18.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836565 2026-02-19T13:19:18.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:18.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:18.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836565 2026-02-19T13:19:18.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:18.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836565 2026-02-19T13:19:18.840 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836565 2026-02-19T13:19:18.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836565' 2026-02-19T13:19:18.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:18.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836563 -lt 21474836565 2026-02-19T13:19:18.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:19.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:19.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:20.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836566 -lt 21474836565 2026-02-19T13:19:20.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:20.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705747 2026-02-19T13:19:20.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:20.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:20.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705747 2026-02-19T13:19:20.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:20.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705747 2026-02-19T13:19:20.161 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705747 2026-02-19T13:19:20.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705747' 2026-02-19T13:19:20.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:20.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705747 -lt 38654705747 2026-02-19T13:19:20.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:20.331 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574928 2026-02-19T13:19:20.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:20.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:20.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574928 2026-02-19T13:19:20.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:20.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574928 2026-02-19T13:19:20.335 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574928 2026-02-19T13:19:20.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574928' 2026-02-19T13:19:20.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:20.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574929 -lt 55834574928 2026-02-19T13:19:20.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:20.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:20.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:20.649 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:20.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:20.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:20.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:20.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:20.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:20.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:20.817 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:20.817 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:20.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:20.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:20.817 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:20.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836568 2026-02-19T13:19:20.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836568 2026-02-19T13:19:20.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836568' 2026-02-19T13:19:20.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:20.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:20.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705750 2026-02-19T13:19:20.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705750 2026-02-19T13:19:20.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836568 1-38654705750' 2026-02-19T13:19:20.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:20.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:21.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574931 2026-02-19T13:19:21.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574931 2026-02-19T13:19:21.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836568 1-38654705750 2-55834574931' 2026-02-19T13:19:21.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:21.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836568 2026-02-19T13:19:21.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:21.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:21.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836568 2026-02-19T13:19:21.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:21.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836568 2026-02-19T13:19:21.035 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836568 2026-02-19T13:19:21.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836568' 2026-02-19T13:19:21.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:21.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836566 -lt 21474836568 2026-02-19T13:19:21.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:22.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:22.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:22.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836569 -lt 21474836568 2026-02-19T13:19:22.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:22.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705750 2026-02-19T13:19:22.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:22.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:22.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705750 2026-02-19T13:19:22.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:22.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705750 2026-02-19T13:19:22.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705750' 2026-02-19T13:19:22.360 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705750 2026-02-19T13:19:22.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:22.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705750 -lt 38654705750 2026-02-19T13:19:22.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:22.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574931 2026-02-19T13:19:22.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:22.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:22.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574931 2026-02-19T13:19:22.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:22.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574931 2026-02-19T13:19:22.524 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574931 2026-02-19T13:19:22.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574931' 2026-02-19T13:19:22.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:22.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574932 -lt 55834574931 2026-02-19T13:19:22.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:22.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:22.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:22.827 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:22.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:22.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:22.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:22.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:22.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:22.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:22.991 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:22.991 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:22.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:22.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:22.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:23.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836571 2026-02-19T13:19:23.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836571 2026-02-19T13:19:23.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836571' 2026-02-19T13:19:23.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:23.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:23.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705753 2026-02-19T13:19:23.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705753 2026-02-19T13:19:23.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836571 1-38654705753' 2026-02-19T13:19:23.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:23.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:23.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574935 2026-02-19T13:19:23.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574935 2026-02-19T13:19:23.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836571 1-38654705753 2-55834574935' 2026-02-19T13:19:23.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:23.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836571 2026-02-19T13:19:23.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:23.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:23.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836571 2026-02-19T13:19:23.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:23.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836571 2026-02-19T13:19:23.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836571' 2026-02-19T13:19:23.210 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836571 2026-02-19T13:19:23.210 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:23.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836572 -lt 21474836571 2026-02-19T13:19:23.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:23.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705753 2026-02-19T13:19:23.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:23.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:23.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705753 2026-02-19T13:19:23.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:23.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705753 2026-02-19T13:19:23.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705753' 2026-02-19T13:19:23.380 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705753 2026-02-19T13:19:23.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:23.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705753 -lt 38654705753 2026-02-19T13:19:23.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:23.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574935 2026-02-19T13:19:23.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:23.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:23.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574935 2026-02-19T13:19:23.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:23.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574935 2026-02-19T13:19:23.547 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574935 2026-02-19T13:19:23.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574935' 2026-02-19T13:19:23.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:23.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574935 -lt 55834574935 2026-02-19T13:19:23.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:23.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:23.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:23.856 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:23.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:23.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:23.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:23.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:23.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:24.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:24.027 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:24.027 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:24.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:24.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:24.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:24.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836573 2026-02-19T13:19:24.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836573 2026-02-19T13:19:24.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836573' 2026-02-19T13:19:24.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:24.098 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:24.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705755 2026-02-19T13:19:24.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705755 2026-02-19T13:19:24.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836573 1-38654705755' 2026-02-19T13:19:24.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:24.172 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:24.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574937 2026-02-19T13:19:24.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574937 2026-02-19T13:19:24.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836573 1-38654705755 2-55834574937' 2026-02-19T13:19:24.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:24.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836573 2026-02-19T13:19:24.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:24.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:24.245 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836573 2026-02-19T13:19:24.245 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:24.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836573 2026-02-19T13:19:24.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836573' 2026-02-19T13:19:24.246 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836573 2026-02-19T13:19:24.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:24.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836572 -lt 21474836573 2026-02-19T13:19:24.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:25.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:25.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:25.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836575 -lt 21474836573 2026-02-19T13:19:25.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:25.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705755 2026-02-19T13:19:25.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:25.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:25.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705755 2026-02-19T13:19:25.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:25.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705755 2026-02-19T13:19:25.574 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705755 2026-02-19T13:19:25.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705755' 2026-02-19T13:19:25.574 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:25.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705756 -lt 38654705755 2026-02-19T13:19:25.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:25.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574937 2026-02-19T13:19:25.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:25.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:25.744 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574937 2026-02-19T13:19:25.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:25.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574937 2026-02-19T13:19:25.746 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574937 2026-02-19T13:19:25.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574937' 2026-02-19T13:19:25.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:25.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574938 -lt 55834574937 2026-02-19T13:19:25.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:25.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:25.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:26.068 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:26.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:26.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:26.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:26.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:26.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:26.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:26.235 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:26.235 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:26.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:26.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:26.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:26.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836577 2026-02-19T13:19:26.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836577 2026-02-19T13:19:26.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836577' 2026-02-19T13:19:26.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:26.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:26.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705758 2026-02-19T13:19:26.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705758 2026-02-19T13:19:26.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836577 1-38654705758' 2026-02-19T13:19:26.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:26.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:26.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574940 2026-02-19T13:19:26.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574940 2026-02-19T13:19:26.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836577 1-38654705758 2-55834574940' 2026-02-19T13:19:26.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:26.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836577 2026-02-19T13:19:26.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:26.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:26.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836577 2026-02-19T13:19:26.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:26.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836577 2026-02-19T13:19:26.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836577' 2026-02-19T13:19:26.459 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836577 2026-02-19T13:19:26.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:26.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836575 -lt 21474836577 2026-02-19T13:19:26.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:27.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:27.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:27.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836578 -lt 21474836577 2026-02-19T13:19:27.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:27.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705758 2026-02-19T13:19:27.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:27.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:27.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705758 2026-02-19T13:19:27.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:27.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705758 2026-02-19T13:19:27.779 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705758 2026-02-19T13:19:27.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705758' 2026-02-19T13:19:27.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:27.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705759 -lt 38654705758 2026-02-19T13:19:27.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:27.943 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574940 2026-02-19T13:19:27.943 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:27.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:27.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574940 2026-02-19T13:19:27.945 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:27.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574940 2026-02-19T13:19:27.947 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574940 2026-02-19T13:19:27.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574940' 2026-02-19T13:19:27.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:28.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574941 -lt 55834574940 2026-02-19T13:19:28.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:28.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:28.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:28.266 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:28.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:28.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:28.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:28.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:28.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:28.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:28.429 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:28.429 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:28.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:28.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:28.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:28.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836580 2026-02-19T13:19:28.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836580 2026-02-19T13:19:28.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836580' 2026-02-19T13:19:28.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:28.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:28.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705762 2026-02-19T13:19:28.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705762 2026-02-19T13:19:28.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836580 1-38654705762' 2026-02-19T13:19:28.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:28.570 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:28.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574943 2026-02-19T13:19:28.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574943 2026-02-19T13:19:28.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836580 1-38654705762 2-55834574943' 2026-02-19T13:19:28.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:28.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836580 2026-02-19T13:19:28.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:28.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:28.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836580 2026-02-19T13:19:28.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:28.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836580 2026-02-19T13:19:28.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836580' 2026-02-19T13:19:28.646 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836580 2026-02-19T13:19:28.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:28.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836578 -lt 21474836580 2026-02-19T13:19:28.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:29.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:29.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:29.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836581 -lt 21474836580 2026-02-19T13:19:29.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:29.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705762 2026-02-19T13:19:29.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:29.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:29.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705762 2026-02-19T13:19:29.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:29.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705762 2026-02-19T13:19:29.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705762' 2026-02-19T13:19:29.976 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705762 2026-02-19T13:19:29.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:30.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705762 -lt 38654705762 2026-02-19T13:19:30.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:30.139 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574943 2026-02-19T13:19:30.139 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:30.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:30.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574943 2026-02-19T13:19:30.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:30.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574943 2026-02-19T13:19:30.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574943' 2026-02-19T13:19:30.142 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574943 2026-02-19T13:19:30.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:30.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574944 -lt 55834574943 2026-02-19T13:19:30.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:30.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:30.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:30.457 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:30.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:30.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:30.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:30.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:30.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:30.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:30.629 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:30.629 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:30.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:30.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:30.630 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:30.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836583 2026-02-19T13:19:30.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836583 2026-02-19T13:19:30.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836583' 2026-02-19T13:19:30.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:30.702 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:30.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705765 2026-02-19T13:19:30.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705765 2026-02-19T13:19:30.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836583 1-38654705765' 2026-02-19T13:19:30.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:30.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:30.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574946 2026-02-19T13:19:30.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574946 2026-02-19T13:19:30.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836583 1-38654705765 2-55834574946' 2026-02-19T13:19:30.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:30.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836583 2026-02-19T13:19:30.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:30.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:30.849 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836583 2026-02-19T13:19:30.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:30.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836583 2026-02-19T13:19:30.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836583' 2026-02-19T13:19:30.852 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836583 2026-02-19T13:19:30.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:31.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836581 -lt 21474836583 2026-02-19T13:19:31.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:32.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:32.008 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:32.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836584 -lt 21474836583 2026-02-19T13:19:32.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:32.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705765 2026-02-19T13:19:32.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:32.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:32.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705765 2026-02-19T13:19:32.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:32.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705765 2026-02-19T13:19:32.175 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705765 2026-02-19T13:19:32.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705765' 2026-02-19T13:19:32.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:32.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705765 -lt 38654705765 2026-02-19T13:19:32.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:32.342 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574946 2026-02-19T13:19:32.342 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:32.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:32.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574946 2026-02-19T13:19:32.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:32.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574946 2026-02-19T13:19:32.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574946' 2026-02-19T13:19:32.346 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574946 2026-02-19T13:19:32.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:32.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574947 -lt 55834574946 2026-02-19T13:19:32.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:32.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:32.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:32.659 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:32.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:32.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:32.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:32.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:32.671 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:32.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:32.828 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:32.828 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:32.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:32.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:32.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:32.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836586 2026-02-19T13:19:32.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836586 2026-02-19T13:19:32.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836586' 2026-02-19T13:19:32.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:32.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:32.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705768 2026-02-19T13:19:32.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705768 2026-02-19T13:19:32.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836586 1-38654705768' 2026-02-19T13:19:32.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:32.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:33.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574950 2026-02-19T13:19:33.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574950 2026-02-19T13:19:33.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836586 1-38654705768 2-55834574950' 2026-02-19T13:19:33.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:33.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836586 2026-02-19T13:19:33.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:33.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:33.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836586 2026-02-19T13:19:33.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:33.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836586 2026-02-19T13:19:33.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836586' 2026-02-19T13:19:33.049 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836586 2026-02-19T13:19:33.049 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:33.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836584 -lt 21474836586 2026-02-19T13:19:33.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:34.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:34.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:34.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836587 -lt 21474836586 2026-02-19T13:19:34.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:34.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705768 2026-02-19T13:19:34.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:34.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:34.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705768 2026-02-19T13:19:34.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:34.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705768 2026-02-19T13:19:34.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705768' 2026-02-19T13:19:34.382 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705768 2026-02-19T13:19:34.382 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:34.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705768 -lt 38654705768 2026-02-19T13:19:34.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:34.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574950 2026-02-19T13:19:34.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:34.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:34.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574950 2026-02-19T13:19:34.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:34.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574950 2026-02-19T13:19:34.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574950' 2026-02-19T13:19:34.545 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574950 2026-02-19T13:19:34.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:34.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574950 -lt 55834574950 2026-02-19T13:19:34.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:34.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:34.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:34.856 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:34.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:34.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:34.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:34.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:34.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:35.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:35.029 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:35.029 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:35.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:35.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:35.029 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:35.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836589 2026-02-19T13:19:35.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836589 2026-02-19T13:19:35.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836589' 2026-02-19T13:19:35.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:35.100 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:35.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705771 2026-02-19T13:19:35.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705771 2026-02-19T13:19:35.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836589 1-38654705771' 2026-02-19T13:19:35.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:35.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:35.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574953 2026-02-19T13:19:35.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574953 2026-02-19T13:19:35.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836589 1-38654705771 2-55834574953' 2026-02-19T13:19:35.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:35.245 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836589 2026-02-19T13:19:35.245 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:35.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:35.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836589 2026-02-19T13:19:35.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:35.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836589 2026-02-19T13:19:35.248 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836589 2026-02-19T13:19:35.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836589' 2026-02-19T13:19:35.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:35.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836590 -lt 21474836589 2026-02-19T13:19:35.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:35.416 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705771 2026-02-19T13:19:35.416 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:35.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:35.418 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705771 2026-02-19T13:19:35.418 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:35.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705771 2026-02-19T13:19:35.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705771' 2026-02-19T13:19:35.420 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705771 2026-02-19T13:19:35.420 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:35.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705771 -lt 38654705771 2026-02-19T13:19:35.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:35.577 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574953 2026-02-19T13:19:35.577 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:35.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:35.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574953 2026-02-19T13:19:35.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:35.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574953 2026-02-19T13:19:35.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574953' 2026-02-19T13:19:35.580 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574953 2026-02-19T13:19:35.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:35.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574953 -lt 55834574953 2026-02-19T13:19:35.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:35.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:35.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:35.890 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:35.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:35.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:35.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:35.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:35.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:36.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:36.058 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:36.058 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:36.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:36.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:36.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:36.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836591 2026-02-19T13:19:36.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836591 2026-02-19T13:19:36.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836591' 2026-02-19T13:19:36.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:36.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:36.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705773 2026-02-19T13:19:36.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705773 2026-02-19T13:19:36.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836591 1-38654705773' 2026-02-19T13:19:36.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:36.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:36.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574955 2026-02-19T13:19:36.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574955 2026-02-19T13:19:36.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836591 1-38654705773 2-55834574955' 2026-02-19T13:19:36.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:36.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836591 2026-02-19T13:19:36.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:36.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:36.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836591 2026-02-19T13:19:36.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:36.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836591 2026-02-19T13:19:36.281 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836591 2026-02-19T13:19:36.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836591' 2026-02-19T13:19:36.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:36.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836590 -lt 21474836591 2026-02-19T13:19:36.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:37.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:37.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:37.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836593 -lt 21474836591 2026-02-19T13:19:37.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:37.610 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705773 2026-02-19T13:19:37.610 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:37.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:37.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705773 2026-02-19T13:19:37.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:37.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705773 2026-02-19T13:19:37.613 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705773 2026-02-19T13:19:37.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705773' 2026-02-19T13:19:37.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:37.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705774 -lt 38654705773 2026-02-19T13:19:37.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:37.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574955 2026-02-19T13:19:37.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:37.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:37.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574955 2026-02-19T13:19:37.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:37.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574955 2026-02-19T13:19:37.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574955' 2026-02-19T13:19:37.783 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574955 2026-02-19T13:19:37.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:37.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574956 -lt 55834574955 2026-02-19T13:19:37.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:37.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:37.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:38.099 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:38.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:38.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:38.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:38.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:38.109 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:38.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:38.264 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:38.264 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:38.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:38.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:38.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:38.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836595 2026-02-19T13:19:38.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836595 2026-02-19T13:19:38.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836595' 2026-02-19T13:19:38.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:38.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:38.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705776 2026-02-19T13:19:38.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705776 2026-02-19T13:19:38.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836595 1-38654705776' 2026-02-19T13:19:38.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:38.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:38.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574958 2026-02-19T13:19:38.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574958 2026-02-19T13:19:38.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836595 1-38654705776 2-55834574958' 2026-02-19T13:19:38.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:38.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836595 2026-02-19T13:19:38.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:38.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:38.485 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836595 2026-02-19T13:19:38.485 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:38.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836595 2026-02-19T13:19:38.487 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836595 2026-02-19T13:19:38.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836595' 2026-02-19T13:19:38.488 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:38.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836593 -lt 21474836595 2026-02-19T13:19:38.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:39.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:39.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:39.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836596 -lt 21474836595 2026-02-19T13:19:39.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:39.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705776 2026-02-19T13:19:39.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:39.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:39.811 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705776 2026-02-19T13:19:39.811 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:39.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705776 2026-02-19T13:19:39.813 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705776 2026-02-19T13:19:39.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705776' 2026-02-19T13:19:39.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:39.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705777 -lt 38654705776 2026-02-19T13:19:39.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:39.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574958 2026-02-19T13:19:39.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:39.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:39.987 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574958 2026-02-19T13:19:39.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:39.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574958 2026-02-19T13:19:39.989 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574958 2026-02-19T13:19:39.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574958' 2026-02-19T13:19:39.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:40.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574959 -lt 55834574958 2026-02-19T13:19:40.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:40.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:40.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:40.305 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:40.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:40.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:40.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:40.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:40.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:40.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:40.473 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:40.473 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:40.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:40.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:40.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:40.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836598 2026-02-19T13:19:40.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836598 2026-02-19T13:19:40.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836598' 2026-02-19T13:19:40.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:40.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:40.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705780 2026-02-19T13:19:40.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705780 2026-02-19T13:19:40.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836598 1-38654705780' 2026-02-19T13:19:40.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:40.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:40.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574961 2026-02-19T13:19:40.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574961 2026-02-19T13:19:40.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836598 1-38654705780 2-55834574961' 2026-02-19T13:19:40.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:40.690 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836598 2026-02-19T13:19:40.690 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:40.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:40.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836598 2026-02-19T13:19:40.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:40.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836598 2026-02-19T13:19:40.693 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836598 2026-02-19T13:19:40.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836598' 2026-02-19T13:19:40.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:40.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836596 -lt 21474836598 2026-02-19T13:19:40.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:41.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:41.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:42.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836599 -lt 21474836598 2026-02-19T13:19:42.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:42.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705780 2026-02-19T13:19:42.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:42.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:42.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705780 2026-02-19T13:19:42.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:42.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705780 2026-02-19T13:19:42.024 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705780 2026-02-19T13:19:42.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705780' 2026-02-19T13:19:42.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:42.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705780 -lt 38654705780 2026-02-19T13:19:42.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:42.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574961 2026-02-19T13:19:42.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:42.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:42.185 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574961 2026-02-19T13:19:42.185 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:42.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574961 2026-02-19T13:19:42.187 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574961 2026-02-19T13:19:42.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574961' 2026-02-19T13:19:42.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:42.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574962 -lt 55834574961 2026-02-19T13:19:42.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:42.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:42.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:42.492 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:42.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:42.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:42.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:42.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:42.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:42.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:42.659 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:42.659 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:42.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:42.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:42.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:42.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836601 2026-02-19T13:19:42.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836601 2026-02-19T13:19:42.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836601' 2026-02-19T13:19:42.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:42.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:42.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705783 2026-02-19T13:19:42.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705783 2026-02-19T13:19:42.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836601 1-38654705783' 2026-02-19T13:19:42.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:42.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:42.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574964 2026-02-19T13:19:42.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574964 2026-02-19T13:19:42.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836601 1-38654705783 2-55834574964' 2026-02-19T13:19:42.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:42.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836601 2026-02-19T13:19:42.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:42.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:42.876 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836601 2026-02-19T13:19:42.876 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:42.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836601 2026-02-19T13:19:42.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836601' 2026-02-19T13:19:42.878 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836601 2026-02-19T13:19:42.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:43.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836599 -lt 21474836601 2026-02-19T13:19:43.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:44.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:44.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:44.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836602 -lt 21474836601 2026-02-19T13:19:44.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:44.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705783 2026-02-19T13:19:44.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:44.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:44.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705783 2026-02-19T13:19:44.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:44.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705783 2026-02-19T13:19:44.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705783' 2026-02-19T13:19:44.202 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705783 2026-02-19T13:19:44.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:44.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705783 -lt 38654705783 2026-02-19T13:19:44.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:44.371 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574964 2026-02-19T13:19:44.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:44.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:44.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574964 2026-02-19T13:19:44.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:44.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574964 2026-02-19T13:19:44.375 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574964 2026-02-19T13:19:44.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574964' 2026-02-19T13:19:44.376 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:44.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574965 -lt 55834574964 2026-02-19T13:19:44.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:44.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:44.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:44.682 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:44.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:44.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:44.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:44.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:44.692 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:44.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:44.846 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:44.846 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:44.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:44.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:44.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:44.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836604 2026-02-19T13:19:44.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836604 2026-02-19T13:19:44.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836604' 2026-02-19T13:19:44.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:44.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:44.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705786 2026-02-19T13:19:44.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705786 2026-02-19T13:19:44.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836604 1-38654705786' 2026-02-19T13:19:44.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:44.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:45.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574968 2026-02-19T13:19:45.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574968 2026-02-19T13:19:45.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836604 1-38654705786 2-55834574968' 2026-02-19T13:19:45.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:45.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836604 2026-02-19T13:19:45.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:45.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:45.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836604 2026-02-19T13:19:45.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:45.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836604 2026-02-19T13:19:45.066 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836604 2026-02-19T13:19:45.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836604' 2026-02-19T13:19:45.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:45.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836602 -lt 21474836604 2026-02-19T13:19:45.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:46.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:46.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:46.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836605 -lt 21474836604 2026-02-19T13:19:46.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:46.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705786 2026-02-19T13:19:46.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:46.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:46.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705786 2026-02-19T13:19:46.395 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:46.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705786 2026-02-19T13:19:46.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705786' 2026-02-19T13:19:46.397 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705786 2026-02-19T13:19:46.398 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:46.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705786 -lt 38654705786 2026-02-19T13:19:46.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:46.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574968 2026-02-19T13:19:46.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:46.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:46.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574968 2026-02-19T13:19:46.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:46.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574968 2026-02-19T13:19:46.569 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574968 2026-02-19T13:19:46.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574968' 2026-02-19T13:19:46.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:46.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574968 -lt 55834574968 2026-02-19T13:19:46.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:46.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:46.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:46.887 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:46.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:46.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:46.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:46.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:46.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:47.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:47.055 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:47.056 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:47.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:47.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:47.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:47.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836607 2026-02-19T13:19:47.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836607 2026-02-19T13:19:47.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836607' 2026-02-19T13:19:47.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:47.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:47.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705789 2026-02-19T13:19:47.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705789 2026-02-19T13:19:47.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836607 1-38654705789' 2026-02-19T13:19:47.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:47.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:47.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574971 2026-02-19T13:19:47.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574971 2026-02-19T13:19:47.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836607 1-38654705789 2-55834574971' 2026-02-19T13:19:47.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:47.277 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836607 2026-02-19T13:19:47.277 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:47.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:47.279 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836607 2026-02-19T13:19:47.279 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:47.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836607 2026-02-19T13:19:47.280 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836607 2026-02-19T13:19:47.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836607' 2026-02-19T13:19:47.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:47.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836608 -lt 21474836607 2026-02-19T13:19:47.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:47.444 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705789 2026-02-19T13:19:47.444 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:47.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:47.446 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705789 2026-02-19T13:19:47.446 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:47.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705789 2026-02-19T13:19:47.448 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705789 2026-02-19T13:19:47.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705789' 2026-02-19T13:19:47.448 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:47.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705789 -lt 38654705789 2026-02-19T13:19:47.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:47.607 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574971 2026-02-19T13:19:47.607 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:47.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:47.609 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574971 2026-02-19T13:19:47.609 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:47.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574971 2026-02-19T13:19:47.611 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574971 2026-02-19T13:19:47.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574971' 2026-02-19T13:19:47.611 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:47.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574971 -lt 55834574971 2026-02-19T13:19:47.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:47.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:47.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:47.917 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:47.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:47.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:47.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:47.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:47.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:48.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:48.083 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:48.083 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:48.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:48.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:48.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:48.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836610 2026-02-19T13:19:48.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836610 2026-02-19T13:19:48.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836610' 2026-02-19T13:19:48.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:48.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:48.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705791 2026-02-19T13:19:48.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705791 2026-02-19T13:19:48.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836610 1-38654705791' 2026-02-19T13:19:48.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:48.230 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:48.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574973 2026-02-19T13:19:48.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574973 2026-02-19T13:19:48.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836610 1-38654705791 2-55834574973' 2026-02-19T13:19:48.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:48.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836610 2026-02-19T13:19:48.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:48.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:48.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836610 2026-02-19T13:19:48.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:48.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836610 2026-02-19T13:19:48.305 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836610 2026-02-19T13:19:48.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836610' 2026-02-19T13:19:48.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:48.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836608 -lt 21474836610 2026-02-19T13:19:48.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:49.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:49.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:49.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836611 -lt 21474836610 2026-02-19T13:19:49.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:49.635 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705791 2026-02-19T13:19:49.635 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:49.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:49.637 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705791 2026-02-19T13:19:49.637 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:49.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705791 2026-02-19T13:19:49.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705791' 2026-02-19T13:19:49.639 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705791 2026-02-19T13:19:49.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:49.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705792 -lt 38654705791 2026-02-19T13:19:49.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:49.806 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574973 2026-02-19T13:19:49.806 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:49.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:49.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574973 2026-02-19T13:19:49.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:49.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574973 2026-02-19T13:19:49.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574973' 2026-02-19T13:19:49.809 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574973 2026-02-19T13:19:49.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:49.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574974 -lt 55834574973 2026-02-19T13:19:49.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:49.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:49.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:50.129 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:50.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:50.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:50.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:50.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:50.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:50.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:50.298 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:50.298 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:50.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:50.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:50.298 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:50.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836613 2026-02-19T13:19:50.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836613 2026-02-19T13:19:50.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836613' 2026-02-19T13:19:50.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:50.370 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:50.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705794 2026-02-19T13:19:50.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705794 2026-02-19T13:19:50.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836613 1-38654705794' 2026-02-19T13:19:50.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:50.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:50.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574976 2026-02-19T13:19:50.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574976 2026-02-19T13:19:50.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836613 1-38654705794 2-55834574976' 2026-02-19T13:19:50.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:50.517 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836613 2026-02-19T13:19:50.517 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:50.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:50.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836613 2026-02-19T13:19:50.519 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:50.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836613 2026-02-19T13:19:50.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836613' 2026-02-19T13:19:50.521 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836613 2026-02-19T13:19:50.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:50.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836611 -lt 21474836613 2026-02-19T13:19:50.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:51.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:51.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:51.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836614 -lt 21474836613 2026-02-19T13:19:51.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:51.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705794 2026-02-19T13:19:51.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:51.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:51.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705794 2026-02-19T13:19:51.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:51.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705794 2026-02-19T13:19:51.846 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705794 2026-02-19T13:19:51.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705794' 2026-02-19T13:19:51.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:52.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705795 -lt 38654705794 2026-02-19T13:19:52.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:52.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574976 2026-02-19T13:19:52.015 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:52.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:52.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574976 2026-02-19T13:19:52.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:52.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574976 2026-02-19T13:19:52.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574976' 2026-02-19T13:19:52.019 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574976 2026-02-19T13:19:52.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:52.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574977 -lt 55834574976 2026-02-19T13:19:52.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:52.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:52.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:52.329 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:52.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:52.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:52.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:52.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:52.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:52.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:52.494 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:52.495 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:52.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:52.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:52.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:52.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836616 2026-02-19T13:19:52.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836616 2026-02-19T13:19:52.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836616' 2026-02-19T13:19:52.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:52.566 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:52.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705798 2026-02-19T13:19:52.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705798 2026-02-19T13:19:52.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836616 1-38654705798' 2026-02-19T13:19:52.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:52.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:52.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574979 2026-02-19T13:19:52.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574979 2026-02-19T13:19:52.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836616 1-38654705798 2-55834574979' 2026-02-19T13:19:52.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:52.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836616 2026-02-19T13:19:52.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:52.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:52.714 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836616 2026-02-19T13:19:52.714 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:52.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836616 2026-02-19T13:19:52.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836616' 2026-02-19T13:19:52.716 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836616 2026-02-19T13:19:52.716 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:52.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836614 -lt 21474836616 2026-02-19T13:19:52.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:53.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:53.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:54.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836617 -lt 21474836616 2026-02-19T13:19:54.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:54.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705798 2026-02-19T13:19:54.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:54.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:54.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705798 2026-02-19T13:19:54.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:54.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705798 2026-02-19T13:19:54.038 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705798 2026-02-19T13:19:54.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705798' 2026-02-19T13:19:54.039 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:54.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705798 -lt 38654705798 2026-02-19T13:19:54.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:54.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574979 2026-02-19T13:19:54.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:54.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:54.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574979 2026-02-19T13:19:54.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:54.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574979 2026-02-19T13:19:54.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574979' 2026-02-19T13:19:54.208 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574979 2026-02-19T13:19:54.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:54.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574980 -lt 55834574979 2026-02-19T13:19:54.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:54.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:54.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:54.517 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:54.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:54.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:54.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:54.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:54.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:54.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:54.681 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:54.682 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:54.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:54.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:54.682 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:54.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836619 2026-02-19T13:19:54.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836619 2026-02-19T13:19:54.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836619' 2026-02-19T13:19:54.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:54.754 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:54.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705801 2026-02-19T13:19:54.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705801 2026-02-19T13:19:54.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836619 1-38654705801' 2026-02-19T13:19:54.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:54.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:54.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574982 2026-02-19T13:19:54.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574982 2026-02-19T13:19:54.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836619 1-38654705801 2-55834574982' 2026-02-19T13:19:54.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:54.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836619 2026-02-19T13:19:54.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:54.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:54.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836619 2026-02-19T13:19:54.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:54.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836619 2026-02-19T13:19:54.902 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836619 2026-02-19T13:19:54.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836619' 2026-02-19T13:19:54.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:55.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836617 -lt 21474836619 2026-02-19T13:19:55.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:56.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:56.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:56.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836620 -lt 21474836619 2026-02-19T13:19:56.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:56.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705801 2026-02-19T13:19:56.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:56.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:56.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705801 2026-02-19T13:19:56.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:56.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705801 2026-02-19T13:19:56.224 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705801 2026-02-19T13:19:56.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705801' 2026-02-19T13:19:56.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:56.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705801 -lt 38654705801 2026-02-19T13:19:56.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:56.379 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574982 2026-02-19T13:19:56.379 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:56.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:56.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574982 2026-02-19T13:19:56.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:56.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574982 2026-02-19T13:19:56.382 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574982 2026-02-19T13:19:56.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574982' 2026-02-19T13:19:56.383 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:56.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574983 -lt 55834574982 2026-02-19T13:19:56.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:56.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:56.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:56.708 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:56.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:56.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:56.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:56.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:56.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:56.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:56.874 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:56.875 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:56.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:56.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:56.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:56.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836622 2026-02-19T13:19:56.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836622 2026-02-19T13:19:56.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836622' 2026-02-19T13:19:56.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:56.946 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:57.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705804 2026-02-19T13:19:57.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705804 2026-02-19T13:19:57.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836622 1-38654705804' 2026-02-19T13:19:57.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:57.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:57.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574986 2026-02-19T13:19:57.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574986 2026-02-19T13:19:57.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836622 1-38654705804 2-55834574986' 2026-02-19T13:19:57.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:57.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836622 2026-02-19T13:19:57.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:57.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:57.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836622 2026-02-19T13:19:57.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:57.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836622 2026-02-19T13:19:57.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836622' 2026-02-19T13:19:57.094 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836622 2026-02-19T13:19:57.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:57.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836620 -lt 21474836622 2026-02-19T13:19:57.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:58.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:58.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:58.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836623 -lt 21474836622 2026-02-19T13:19:58.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:58.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705804 2026-02-19T13:19:58.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:58.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:58.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705804 2026-02-19T13:19:58.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:58.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705804 2026-02-19T13:19:58.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705804' 2026-02-19T13:19:58.426 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705804 2026-02-19T13:19:58.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:58.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705804 -lt 38654705804 2026-02-19T13:19:58.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:58.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574986 2026-02-19T13:19:58.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:58.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:58.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574986 2026-02-19T13:19:58.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:58.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574986 2026-02-19T13:19:58.597 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574986 2026-02-19T13:19:58.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574986' 2026-02-19T13:19:58.597 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:58.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574986 -lt 55834574986 2026-02-19T13:19:58.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:19:58.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:19:58.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:19:58.928 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:19:58.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:19:58.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:19:58.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:19:58.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:58.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:59.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:59.083 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:19:59.083 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:19:59.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:59.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:59.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:59.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836626 2026-02-19T13:19:59.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836626 2026-02-19T13:19:59.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836626' 2026-02-19T13:19:59.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:59.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:59.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705807 2026-02-19T13:19:59.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705807 2026-02-19T13:19:59.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836626 1-38654705807' 2026-02-19T13:19:59.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:59.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:59.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574989 2026-02-19T13:19:59.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574989 2026-02-19T13:19:59.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836626 1-38654705807 2-55834574989' 2026-02-19T13:19:59.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:59.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836626 2026-02-19T13:19:59.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:59.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:59.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836626 2026-02-19T13:19:59.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:59.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836626 2026-02-19T13:19:59.307 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836626 2026-02-19T13:19:59.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836626' 2026-02-19T13:19:59.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:59.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836626 -lt 21474836626 2026-02-19T13:19:59.475 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:59.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705807 2026-02-19T13:19:59.476 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:59.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:59.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705807 2026-02-19T13:19:59.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:59.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705807 2026-02-19T13:19:59.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705807' 2026-02-19T13:19:59.480 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705807 2026-02-19T13:19:59.480 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:59.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705807 -lt 38654705807 2026-02-19T13:19:59.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:59.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574989 2026-02-19T13:19:59.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:59.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:59.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574989 2026-02-19T13:19:59.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:59.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574989 2026-02-19T13:19:59.646 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574989 2026-02-19T13:19:59.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574989' 2026-02-19T13:19:59.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:59.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574988 -lt 55834574989 2026-02-19T13:19:59.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:00.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:00.811 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:00.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574988 -lt 55834574989 2026-02-19T13:20:00.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:01.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:20:01.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:02.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574991 -lt 55834574989 2026-02-19T13:20:02.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:02.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:02.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:02.282 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:02.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:02.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:02.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:02.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:02.292 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:02.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:02.451 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:02.451 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:02.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:02.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:02.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:02.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836630 2026-02-19T13:20:02.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836630 2026-02-19T13:20:02.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836630' 2026-02-19T13:20:02.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:02.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:02.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705812 2026-02-19T13:20:02.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705812 2026-02-19T13:20:02.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836630 1-38654705812' 2026-02-19T13:20:02.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:02.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:02.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574993 2026-02-19T13:20:02.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574993 2026-02-19T13:20:02.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836630 1-38654705812 2-55834574993' 2026-02-19T13:20:02.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:02.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836630 2026-02-19T13:20:02.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:02.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:02.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836630 2026-02-19T13:20:02.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:02.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836630 2026-02-19T13:20:02.670 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836630 2026-02-19T13:20:02.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836630' 2026-02-19T13:20:02.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:02.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836628 -lt 21474836630 2026-02-19T13:20:02.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:03.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:03.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:03.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836631 -lt 21474836630 2026-02-19T13:20:03.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:03.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705812 2026-02-19T13:20:03.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:03.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:03.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705812 2026-02-19T13:20:03.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:03.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705812 2026-02-19T13:20:04.000 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705812 2026-02-19T13:20:04.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705812' 2026-02-19T13:20:04.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:04.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705812 -lt 38654705812 2026-02-19T13:20:04.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:04.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574993 2026-02-19T13:20:04.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:04.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:04.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574993 2026-02-19T13:20:04.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:04.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574993 2026-02-19T13:20:04.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574993' 2026-02-19T13:20:04.169 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574993 2026-02-19T13:20:04.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:04.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574994 -lt 55834574993 2026-02-19T13:20:04.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:04.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:04.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:04.471 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:04.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:04.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:04.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:04.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:04.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:04.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:04.643 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:04.643 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:04.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:04.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:04.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:04.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836633 2026-02-19T13:20:04.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836633 2026-02-19T13:20:04.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836633' 2026-02-19T13:20:04.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:04.716 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:04.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705815 2026-02-19T13:20:04.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705815 2026-02-19T13:20:04.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836633 1-38654705815' 2026-02-19T13:20:04.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:04.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:04.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574996 2026-02-19T13:20:04.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574996 2026-02-19T13:20:04.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836633 1-38654705815 2-55834574996' 2026-02-19T13:20:04.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:04.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836633 2026-02-19T13:20:04.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:04.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:04.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836633 2026-02-19T13:20:04.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:04.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836633 2026-02-19T13:20:04.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836633' 2026-02-19T13:20:04.867 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836633 2026-02-19T13:20:04.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:05.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836631 -lt 21474836633 2026-02-19T13:20:05.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:06.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:06.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:06.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836634 -lt 21474836633 2026-02-19T13:20:06.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:06.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705815 2026-02-19T13:20:06.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:06.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:06.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705815 2026-02-19T13:20:06.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:06.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705815 2026-02-19T13:20:06.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705815' 2026-02-19T13:20:06.191 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705815 2026-02-19T13:20:06.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:06.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705815 -lt 38654705815 2026-02-19T13:20:06.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:06.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574996 2026-02-19T13:20:06.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:06.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:06.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574996 2026-02-19T13:20:06.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:06.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574996 2026-02-19T13:20:06.358 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574996 2026-02-19T13:20:06.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574996' 2026-02-19T13:20:06.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:06.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574997 -lt 55834574996 2026-02-19T13:20:06.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:06.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:06.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:06.666 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:06.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:06.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:06.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:06.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:06.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:06.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:06.827 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:06.827 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:06.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:06.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:06.827 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:06.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836636 2026-02-19T13:20:06.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836636 2026-02-19T13:20:06.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836636' 2026-02-19T13:20:06.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:06.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:06.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705818 2026-02-19T13:20:06.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705818 2026-02-19T13:20:06.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836636 1-38654705818' 2026-02-19T13:20:06.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:06.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:07.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574999 2026-02-19T13:20:07.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574999 2026-02-19T13:20:07.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836636 1-38654705818 2-55834574999' 2026-02-19T13:20:07.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:07.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836636 2026-02-19T13:20:07.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:07.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:07.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836636 2026-02-19T13:20:07.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:07.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836636 2026-02-19T13:20:07.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836636' 2026-02-19T13:20:07.045 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836636 2026-02-19T13:20:07.045 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:07.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836634 -lt 21474836636 2026-02-19T13:20:07.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:08.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:08.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:08.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836637 -lt 21474836636 2026-02-19T13:20:08.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:08.370 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705818 2026-02-19T13:20:08.370 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:08.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:08.371 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705818 2026-02-19T13:20:08.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:08.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705818 2026-02-19T13:20:08.373 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705818 2026-02-19T13:20:08.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705818' 2026-02-19T13:20:08.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:08.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705818 -lt 38654705818 2026-02-19T13:20:08.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:08.538 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574999 2026-02-19T13:20:08.539 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:08.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:08.540 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574999 2026-02-19T13:20:08.540 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:08.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574999 2026-02-19T13:20:08.542 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834574999 2026-02-19T13:20:08.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574999' 2026-02-19T13:20:08.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:08.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575000 -lt 55834574999 2026-02-19T13:20:08.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:08.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:08.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:08.859 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:08.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:08.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:08.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:08.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:08.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:09.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:09.030 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:09.030 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:09.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:09.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:09.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:09.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836639 2026-02-19T13:20:09.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836639 2026-02-19T13:20:09.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836639' 2026-02-19T13:20:09.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:09.101 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:09.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705821 2026-02-19T13:20:09.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705821 2026-02-19T13:20:09.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836639 1-38654705821' 2026-02-19T13:20:09.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:09.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:09.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575003 2026-02-19T13:20:09.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575003 2026-02-19T13:20:09.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836639 1-38654705821 2-55834575003' 2026-02-19T13:20:09.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:09.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836639 2026-02-19T13:20:09.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:09.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:09.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836639 2026-02-19T13:20:09.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:09.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836639 2026-02-19T13:20:09.249 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836639 2026-02-19T13:20:09.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836639' 2026-02-19T13:20:09.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:09.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836640 -lt 21474836639 2026-02-19T13:20:09.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:09.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705821 2026-02-19T13:20:09.419 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:09.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:09.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705821 2026-02-19T13:20:09.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:09.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705821 2026-02-19T13:20:09.423 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705821 2026-02-19T13:20:09.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705821' 2026-02-19T13:20:09.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:09.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705821 -lt 38654705821 2026-02-19T13:20:09.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:09.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575003 2026-02-19T13:20:09.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:09.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:09.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575003 2026-02-19T13:20:09.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:09.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575003 2026-02-19T13:20:09.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575003' 2026-02-19T13:20:09.588 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575003 2026-02-19T13:20:09.589 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:09.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575003 -lt 55834575003 2026-02-19T13:20:09.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:09.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:09.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:09.893 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:09.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:09.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:09.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:09.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:09.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:10.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:10.061 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:10.061 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:10.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:10.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:10.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:10.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836641 2026-02-19T13:20:10.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836641 2026-02-19T13:20:10.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836641' 2026-02-19T13:20:10.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:10.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:10.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705823 2026-02-19T13:20:10.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705823 2026-02-19T13:20:10.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836641 1-38654705823' 2026-02-19T13:20:10.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:10.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:10.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575005 2026-02-19T13:20:10.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575005 2026-02-19T13:20:10.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836641 1-38654705823 2-55834575005' 2026-02-19T13:20:10.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:10.279 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836641 2026-02-19T13:20:10.279 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:10.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:10.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836641 2026-02-19T13:20:10.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:10.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836641 2026-02-19T13:20:10.282 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836641 2026-02-19T13:20:10.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836641' 2026-02-19T13:20:10.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:10.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836640 -lt 21474836641 2026-02-19T13:20:10.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:11.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:11.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:11.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836643 -lt 21474836641 2026-02-19T13:20:11.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:11.609 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705823 2026-02-19T13:20:11.609 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:11.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:11.611 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705823 2026-02-19T13:20:11.611 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:11.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705823 2026-02-19T13:20:11.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705823' 2026-02-19T13:20:11.613 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705823 2026-02-19T13:20:11.613 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:11.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705824 -lt 38654705823 2026-02-19T13:20:11.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:11.785 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575005 2026-02-19T13:20:11.785 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:11.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:11.787 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575005 2026-02-19T13:20:11.787 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:11.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575005 2026-02-19T13:20:11.788 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575005 2026-02-19T13:20:11.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575005' 2026-02-19T13:20:11.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:11.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575006 -lt 55834575005 2026-02-19T13:20:11.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:11.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:11.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:12.101 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:12.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:12.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:12.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:12.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:12.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:12.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:12.272 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:12.272 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:12.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:12.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:12.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:12.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836645 2026-02-19T13:20:12.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836645 2026-02-19T13:20:12.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836645' 2026-02-19T13:20:12.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:12.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:12.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705826 2026-02-19T13:20:12.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705826 2026-02-19T13:20:12.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836645 1-38654705826' 2026-02-19T13:20:12.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:12.418 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:12.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575008 2026-02-19T13:20:12.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575008 2026-02-19T13:20:12.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836645 1-38654705826 2-55834575008' 2026-02-19T13:20:12.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:12.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836645 2026-02-19T13:20:12.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:12.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:12.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836645 2026-02-19T13:20:12.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:12.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836645 2026-02-19T13:20:12.492 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836645 2026-02-19T13:20:12.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836645' 2026-02-19T13:20:12.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:12.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836643 -lt 21474836645 2026-02-19T13:20:12.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:13.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:13.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:13.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836646 -lt 21474836645 2026-02-19T13:20:13.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:13.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705826 2026-02-19T13:20:13.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:13.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:13.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705826 2026-02-19T13:20:13.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:13.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705826 2026-02-19T13:20:13.825 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705826 2026-02-19T13:20:13.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705826' 2026-02-19T13:20:13.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:13.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705827 -lt 38654705826 2026-02-19T13:20:13.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:13.995 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575008 2026-02-19T13:20:13.995 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:13.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:13.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575008 2026-02-19T13:20:13.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:13.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575008 2026-02-19T13:20:13.998 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575008 2026-02-19T13:20:13.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575008' 2026-02-19T13:20:13.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:14.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575009 -lt 55834575008 2026-02-19T13:20:14.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:14.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:14.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:14.312 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:14.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:14.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:14.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:14.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:14.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:14.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:14.477 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:14.477 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:14.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:14.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:14.477 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:14.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836648 2026-02-19T13:20:14.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836648 2026-02-19T13:20:14.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836648' 2026-02-19T13:20:14.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:14.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:14.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705830 2026-02-19T13:20:14.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705830 2026-02-19T13:20:14.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836648 1-38654705830' 2026-02-19T13:20:14.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:14.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:14.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575011 2026-02-19T13:20:14.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575011 2026-02-19T13:20:14.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836648 1-38654705830 2-55834575011' 2026-02-19T13:20:14.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:14.701 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836648 2026-02-19T13:20:14.701 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:14.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:14.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836648 2026-02-19T13:20:14.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:14.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836648 2026-02-19T13:20:14.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836648' 2026-02-19T13:20:14.705 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836648 2026-02-19T13:20:14.705 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:14.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836646 -lt 21474836648 2026-02-19T13:20:14.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:15.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:15.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:16.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836649 -lt 21474836648 2026-02-19T13:20:16.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:16.029 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705830 2026-02-19T13:20:16.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:16.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:16.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705830 2026-02-19T13:20:16.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:16.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705830 2026-02-19T13:20:16.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705830' 2026-02-19T13:20:16.033 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705830 2026-02-19T13:20:16.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:16.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705830 -lt 38654705830 2026-02-19T13:20:16.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:16.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575011 2026-02-19T13:20:16.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:16.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:16.200 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575011 2026-02-19T13:20:16.200 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:16.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575011 2026-02-19T13:20:16.201 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575011 2026-02-19T13:20:16.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575011' 2026-02-19T13:20:16.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:16.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575012 -lt 55834575011 2026-02-19T13:20:16.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:16.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:16.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:16.512 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:16.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:16.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:16.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:16.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:16.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:16.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:16.682 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:16.682 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:16.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:16.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:16.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:16.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836651 2026-02-19T13:20:16.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836651 2026-02-19T13:20:16.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836651' 2026-02-19T13:20:16.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:16.755 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:16.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705833 2026-02-19T13:20:16.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705833 2026-02-19T13:20:16.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836651 1-38654705833' 2026-02-19T13:20:16.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:16.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:16.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575014 2026-02-19T13:20:16.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575014 2026-02-19T13:20:16.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836651 1-38654705833 2-55834575014' 2026-02-19T13:20:16.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:16.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836651 2026-02-19T13:20:16.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:16.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:16.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836651 2026-02-19T13:20:16.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:16.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836651 2026-02-19T13:20:16.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836651' 2026-02-19T13:20:16.900 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836651 2026-02-19T13:20:16.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:17.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836649 -lt 21474836651 2026-02-19T13:20:17.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:18.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:18.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:18.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836652 -lt 21474836651 2026-02-19T13:20:18.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:18.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705833 2026-02-19T13:20:18.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:18.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:18.237 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705833 2026-02-19T13:20:18.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:18.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705833 2026-02-19T13:20:18.239 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705833 2026-02-19T13:20:18.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705833' 2026-02-19T13:20:18.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:18.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705833 -lt 38654705833 2026-02-19T13:20:18.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:18.396 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575014 2026-02-19T13:20:18.397 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:18.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:18.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575014 2026-02-19T13:20:18.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:18.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575014 2026-02-19T13:20:18.400 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575014 2026-02-19T13:20:18.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575014' 2026-02-19T13:20:18.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:18.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575015 -lt 55834575014 2026-02-19T13:20:18.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:18.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:18.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:18.709 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:18.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:18.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:18.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:18.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:18.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:18.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:18.880 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:18.880 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:18.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:18.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:18.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:18.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836654 2026-02-19T13:20:18.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836654 2026-02-19T13:20:18.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836654' 2026-02-19T13:20:18.951 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:18.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:19.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705836 2026-02-19T13:20:19.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705836 2026-02-19T13:20:19.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836654 1-38654705836' 2026-02-19T13:20:19.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:19.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:19.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575018 2026-02-19T13:20:19.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575018 2026-02-19T13:20:19.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836654 1-38654705836 2-55834575018' 2026-02-19T13:20:19.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:19.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836654 2026-02-19T13:20:19.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:19.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:19.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836654 2026-02-19T13:20:19.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:19.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836654 2026-02-19T13:20:19.096 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836654 2026-02-19T13:20:19.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836654' 2026-02-19T13:20:19.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:19.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836652 -lt 21474836654 2026-02-19T13:20:19.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:20.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:20.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:20.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836655 -lt 21474836654 2026-02-19T13:20:20.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:20.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705836 2026-02-19T13:20:20.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:20.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:20.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705836 2026-02-19T13:20:20.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:20.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705836 2026-02-19T13:20:20.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705836' 2026-02-19T13:20:20.425 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705836 2026-02-19T13:20:20.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:20.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705836 -lt 38654705836 2026-02-19T13:20:20.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:20.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575018 2026-02-19T13:20:20.600 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:20.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:20.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575018 2026-02-19T13:20:20.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:20.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575018 2026-02-19T13:20:20.603 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575018 2026-02-19T13:20:20.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575018' 2026-02-19T13:20:20.604 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:20.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575018 -lt 55834575018 2026-02-19T13:20:20.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:20.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:20.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:20.906 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:20.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:20.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:20.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:20.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:20.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:21.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:21.077 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:21.078 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:21.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:21.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:21.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:21.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836657 2026-02-19T13:20:21.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836657 2026-02-19T13:20:21.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836657' 2026-02-19T13:20:21.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:21.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:21.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705839 2026-02-19T13:20:21.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705839 2026-02-19T13:20:21.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836657 1-38654705839' 2026-02-19T13:20:21.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:21.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:21.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575021 2026-02-19T13:20:21.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575021 2026-02-19T13:20:21.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836657 1-38654705839 2-55834575021' 2026-02-19T13:20:21.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:21.293 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836657 2026-02-19T13:20:21.293 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:21.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:21.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836657 2026-02-19T13:20:21.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:21.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836657 2026-02-19T13:20:21.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836657' 2026-02-19T13:20:21.296 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836657 2026-02-19T13:20:21.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:21.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836658 -lt 21474836657 2026-02-19T13:20:21.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:21.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705839 2026-02-19T13:20:21.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:21.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:21.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705839 2026-02-19T13:20:21.452 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:21.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705839 2026-02-19T13:20:21.453 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705839 2026-02-19T13:20:21.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705839' 2026-02-19T13:20:21.454 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:21.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705839 -lt 38654705839 2026-02-19T13:20:21.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:21.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575021 2026-02-19T13:20:21.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:21.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:21.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575021 2026-02-19T13:20:21.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:21.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575021 2026-02-19T13:20:21.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575021' 2026-02-19T13:20:21.625 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575021 2026-02-19T13:20:21.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:21.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575021 -lt 55834575021 2026-02-19T13:20:21.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:21.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:21.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:21.936 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:21.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:21.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:21.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:21.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:21.946 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:22.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:22.103 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:22.103 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:22.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:22.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:22.104 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:22.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836660 2026-02-19T13:20:22.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836660 2026-02-19T13:20:22.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836660' 2026-02-19T13:20:22.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:22.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:22.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705841 2026-02-19T13:20:22.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705841 2026-02-19T13:20:22.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836660 1-38654705841' 2026-02-19T13:20:22.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:22.245 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:22.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575023 2026-02-19T13:20:22.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575023 2026-02-19T13:20:22.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836660 1-38654705841 2-55834575023' 2026-02-19T13:20:22.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:22.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836660 2026-02-19T13:20:22.318 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:22.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:22.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836660 2026-02-19T13:20:22.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:22.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836660 2026-02-19T13:20:22.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836660' 2026-02-19T13:20:22.322 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836660 2026-02-19T13:20:22.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:22.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836658 -lt 21474836660 2026-02-19T13:20:22.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:23.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:23.485 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:23.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836661 -lt 21474836660 2026-02-19T13:20:23.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:23.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705841 2026-02-19T13:20:23.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:23.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:23.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705841 2026-02-19T13:20:23.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:23.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705841 2026-02-19T13:20:23.642 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705841 2026-02-19T13:20:23.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705841' 2026-02-19T13:20:23.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:23.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705842 -lt 38654705841 2026-02-19T13:20:23.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:23.807 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575023 2026-02-19T13:20:23.807 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:23.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:23.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575023 2026-02-19T13:20:23.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:23.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575023 2026-02-19T13:20:23.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575023' 2026-02-19T13:20:23.811 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575023 2026-02-19T13:20:23.811 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:23.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575024 -lt 55834575023 2026-02-19T13:20:23.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:23.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:23.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:24.120 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:24.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:24.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:24.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:24.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:24.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:24.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:24.289 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:24.289 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:24.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:24.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:24.290 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:24.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836663 2026-02-19T13:20:24.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836663 2026-02-19T13:20:24.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836663' 2026-02-19T13:20:24.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:24.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:24.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705844 2026-02-19T13:20:24.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705844 2026-02-19T13:20:24.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836663 1-38654705844' 2026-02-19T13:20:24.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:24.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:24.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575026 2026-02-19T13:20:24.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575026 2026-02-19T13:20:24.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836663 1-38654705844 2-55834575026' 2026-02-19T13:20:24.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:24.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836663 2026-02-19T13:20:24.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:24.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:24.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836663 2026-02-19T13:20:24.509 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:24.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836663 2026-02-19T13:20:24.510 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836663 2026-02-19T13:20:24.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836663' 2026-02-19T13:20:24.511 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:24.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836661 -lt 21474836663 2026-02-19T13:20:24.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:25.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:25.680 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:25.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836664 -lt 21474836663 2026-02-19T13:20:25.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:25.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705844 2026-02-19T13:20:25.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:25.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:25.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705844 2026-02-19T13:20:25.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:25.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705844 2026-02-19T13:20:25.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705844' 2026-02-19T13:20:25.842 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705844 2026-02-19T13:20:25.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:26.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705845 -lt 38654705844 2026-02-19T13:20:26.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:26.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575026 2026-02-19T13:20:26.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:26.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:26.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575026 2026-02-19T13:20:26.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:26.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575026 2026-02-19T13:20:26.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575026' 2026-02-19T13:20:26.007 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575026 2026-02-19T13:20:26.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:26.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575027 -lt 55834575026 2026-02-19T13:20:26.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:26.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:26.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:26.311 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:26.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:26.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:26.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:26.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:26.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:26.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:26.479 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:26.479 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:26.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:26.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:26.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:26.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836666 2026-02-19T13:20:26.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836666 2026-02-19T13:20:26.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836666' 2026-02-19T13:20:26.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:26.553 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:26.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705848 2026-02-19T13:20:26.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705848 2026-02-19T13:20:26.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836666 1-38654705848' 2026-02-19T13:20:26.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:26.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:26.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575029 2026-02-19T13:20:26.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575029 2026-02-19T13:20:26.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836666 1-38654705848 2-55834575029' 2026-02-19T13:20:26.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:26.697 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836666 2026-02-19T13:20:26.697 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:26.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:26.700 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836666 2026-02-19T13:20:26.700 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:26.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836666 2026-02-19T13:20:26.701 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836666 2026-02-19T13:20:26.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836666' 2026-02-19T13:20:26.701 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:26.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836664 -lt 21474836666 2026-02-19T13:20:26.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:27.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:27.873 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:28.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836667 -lt 21474836666 2026-02-19T13:20:28.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:28.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705848 2026-02-19T13:20:28.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:28.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:28.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705848 2026-02-19T13:20:28.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:28.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705848 2026-02-19T13:20:28.036 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705848 2026-02-19T13:20:28.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705848' 2026-02-19T13:20:28.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:28.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705848 -lt 38654705848 2026-02-19T13:20:28.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:28.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575029 2026-02-19T13:20:28.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:28.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:28.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575029 2026-02-19T13:20:28.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:28.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575029 2026-02-19T13:20:28.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575029' 2026-02-19T13:20:28.201 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575029 2026-02-19T13:20:28.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:28.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575030 -lt 55834575029 2026-02-19T13:20:28.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:28.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:28.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:28.522 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:28.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:28.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:28.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:28.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:28.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:28.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:28.692 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:28.692 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:28.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:28.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:28.693 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:28.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836669 2026-02-19T13:20:28.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836669 2026-02-19T13:20:28.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836669' 2026-02-19T13:20:28.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:28.768 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:28.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705851 2026-02-19T13:20:28.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705851 2026-02-19T13:20:28.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836669 1-38654705851' 2026-02-19T13:20:28.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:28.842 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:28.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575032 2026-02-19T13:20:28.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575032 2026-02-19T13:20:28.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836669 1-38654705851 2-55834575032' 2026-02-19T13:20:28.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:28.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836669 2026-02-19T13:20:28.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:28.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:28.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836669 2026-02-19T13:20:28.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:28.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836669 2026-02-19T13:20:28.919 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836669 2026-02-19T13:20:28.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836669' 2026-02-19T13:20:28.920 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:29.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836667 -lt 21474836669 2026-02-19T13:20:29.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:30.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:30.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:30.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836670 -lt 21474836669 2026-02-19T13:20:30.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:30.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705851 2026-02-19T13:20:30.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:30.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:30.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705851 2026-02-19T13:20:30.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:30.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705851 2026-02-19T13:20:30.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705851' 2026-02-19T13:20:30.243 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705851 2026-02-19T13:20:30.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:30.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705851 -lt 38654705851 2026-02-19T13:20:30.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:30.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575032 2026-02-19T13:20:30.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:30.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:30.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575032 2026-02-19T13:20:30.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:30.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575032 2026-02-19T13:20:30.414 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575032 2026-02-19T13:20:30.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575032' 2026-02-19T13:20:30.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:30.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575033 -lt 55834575032 2026-02-19T13:20:30.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:30.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:30.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:30.730 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:30.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:30.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:30.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:30.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:30.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:30.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:30.895 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:30.895 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:30.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:30.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:30.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:30.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836672 2026-02-19T13:20:30.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836672 2026-02-19T13:20:30.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836672' 2026-02-19T13:20:30.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:30.967 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:31.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705854 2026-02-19T13:20:31.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705854 2026-02-19T13:20:31.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836672 1-38654705854' 2026-02-19T13:20:31.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:31.039 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:31.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575036 2026-02-19T13:20:31.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575036 2026-02-19T13:20:31.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836672 1-38654705854 2-55834575036' 2026-02-19T13:20:31.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:31.109 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836672 2026-02-19T13:20:31.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:31.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:31.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836672 2026-02-19T13:20:31.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:31.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836672 2026-02-19T13:20:31.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836672' 2026-02-19T13:20:31.113 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836672 2026-02-19T13:20:31.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:31.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836670 -lt 21474836672 2026-02-19T13:20:31.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:32.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:32.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:32.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836673 -lt 21474836672 2026-02-19T13:20:32.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:32.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705854 2026-02-19T13:20:32.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:32.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:32.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705854 2026-02-19T13:20:32.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:32.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705854 2026-02-19T13:20:32.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705854' 2026-02-19T13:20:32.434 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705854 2026-02-19T13:20:32.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:32.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705854 -lt 38654705854 2026-02-19T13:20:32.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:32.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575036 2026-02-19T13:20:32.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:32.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:32.601 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575036 2026-02-19T13:20:32.601 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:32.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575036 2026-02-19T13:20:32.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575036' 2026-02-19T13:20:32.603 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575036 2026-02-19T13:20:32.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:32.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575036 -lt 55834575036 2026-02-19T13:20:32.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:32.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:32.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:32.924 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:32.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:32.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:32.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:32.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:32.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:33.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:33.091 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:33.091 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:33.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:33.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:33.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:33.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836676 2026-02-19T13:20:33.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836676 2026-02-19T13:20:33.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836676' 2026-02-19T13:20:33.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:33.164 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:33.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705857 2026-02-19T13:20:33.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705857 2026-02-19T13:20:33.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836676 1-38654705857' 2026-02-19T13:20:33.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:33.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:33.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575039 2026-02-19T13:20:33.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575039 2026-02-19T13:20:33.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836676 1-38654705857 2-55834575039' 2026-02-19T13:20:33.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:33.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836676 2026-02-19T13:20:33.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:33.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:33.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836676 2026-02-19T13:20:33.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:33.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836676 2026-02-19T13:20:33.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836676' 2026-02-19T13:20:33.311 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836676 2026-02-19T13:20:33.311 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:33.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836676 -lt 21474836676 2026-02-19T13:20:33.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:33.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705857 2026-02-19T13:20:33.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:33.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:33.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705857 2026-02-19T13:20:33.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:33.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705857 2026-02-19T13:20:33.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705857' 2026-02-19T13:20:33.481 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705857 2026-02-19T13:20:33.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:33.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705857 -lt 38654705857 2026-02-19T13:20:33.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:33.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575039 2026-02-19T13:20:33.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:33.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:33.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575039 2026-02-19T13:20:33.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:33.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575039 2026-02-19T13:20:33.649 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575039 2026-02-19T13:20:33.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575039' 2026-02-19T13:20:33.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:33.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575038 -lt 55834575039 2026-02-19T13:20:33.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:34.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:34.819 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:34.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575038 -lt 55834575039 2026-02-19T13:20:34.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:35.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:20:35.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:36.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575041 -lt 55834575039 2026-02-19T13:20:36.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:36.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:36.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:36.289 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:36.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:36.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:36.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:36.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:36.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:36.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:36.453 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:36.453 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:36.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:36.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:36.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:36.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836680 2026-02-19T13:20:36.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836680 2026-02-19T13:20:36.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836680' 2026-02-19T13:20:36.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:36.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:36.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705862 2026-02-19T13:20:36.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705862 2026-02-19T13:20:36.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836680 1-38654705862' 2026-02-19T13:20:36.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:36.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:36.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575043 2026-02-19T13:20:36.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575043 2026-02-19T13:20:36.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836680 1-38654705862 2-55834575043' 2026-02-19T13:20:36.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:36.673 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836680 2026-02-19T13:20:36.673 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:36.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:36.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836680 2026-02-19T13:20:36.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:36.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836680 2026-02-19T13:20:36.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836680' 2026-02-19T13:20:36.677 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836680 2026-02-19T13:20:36.677 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:36.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836678 -lt 21474836680 2026-02-19T13:20:36.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:37.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:37.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:38.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836681 -lt 21474836680 2026-02-19T13:20:38.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:38.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705862 2026-02-19T13:20:38.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:38.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:38.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705862 2026-02-19T13:20:38.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:38.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705862 2026-02-19T13:20:38.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705862' 2026-02-19T13:20:38.008 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705862 2026-02-19T13:20:38.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:38.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705862 -lt 38654705862 2026-02-19T13:20:38.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:38.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575043 2026-02-19T13:20:38.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:38.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:38.172 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575043 2026-02-19T13:20:38.173 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:38.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575043 2026-02-19T13:20:38.174 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575043 2026-02-19T13:20:38.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575043' 2026-02-19T13:20:38.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:38.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575044 -lt 55834575043 2026-02-19T13:20:38.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:38.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:38.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:38.487 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:38.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:38.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:38.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:38.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:38.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:38.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:38.657 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:38.657 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:38.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:38.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:38.657 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:38.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836683 2026-02-19T13:20:38.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836683 2026-02-19T13:20:38.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836683' 2026-02-19T13:20:38.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:38.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:38.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705865 2026-02-19T13:20:38.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705865 2026-02-19T13:20:38.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836683 1-38654705865' 2026-02-19T13:20:38.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:38.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:38.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575046 2026-02-19T13:20:38.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575046 2026-02-19T13:20:38.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836683 1-38654705865 2-55834575046' 2026-02-19T13:20:38.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:38.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836683 2026-02-19T13:20:38.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:38.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:38.881 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836683 2026-02-19T13:20:38.881 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:38.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836683 2026-02-19T13:20:38.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836683' 2026-02-19T13:20:38.883 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836683 2026-02-19T13:20:38.883 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:39.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836681 -lt 21474836683 2026-02-19T13:20:39.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:40.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:40.054 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:40.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836684 -lt 21474836683 2026-02-19T13:20:40.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:40.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705865 2026-02-19T13:20:40.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:40.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:40.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705865 2026-02-19T13:20:40.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:40.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705865 2026-02-19T13:20:40.213 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705865 2026-02-19T13:20:40.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705865' 2026-02-19T13:20:40.214 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:40.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705865 -lt 38654705865 2026-02-19T13:20:40.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:40.383 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575046 2026-02-19T13:20:40.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:40.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:40.385 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575046 2026-02-19T13:20:40.385 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:40.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575046 2026-02-19T13:20:40.386 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575046 2026-02-19T13:20:40.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575046' 2026-02-19T13:20:40.387 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:40.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575047 -lt 55834575046 2026-02-19T13:20:40.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:40.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:40.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:40.705 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:40.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:40.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:40.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:40.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:40.715 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:40.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:40.878 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:40.878 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:40.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:40.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:40.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:40.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836686 2026-02-19T13:20:40.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836686 2026-02-19T13:20:40.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836686' 2026-02-19T13:20:40.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:40.953 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:41.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705868 2026-02-19T13:20:41.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705868 2026-02-19T13:20:41.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836686 1-38654705868' 2026-02-19T13:20:41.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:41.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:41.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575050 2026-02-19T13:20:41.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575050 2026-02-19T13:20:41.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836686 1-38654705868 2-55834575050' 2026-02-19T13:20:41.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:41.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836686 2026-02-19T13:20:41.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:41.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:41.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836686 2026-02-19T13:20:41.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:41.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836686 2026-02-19T13:20:41.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836686' 2026-02-19T13:20:41.098 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836686 2026-02-19T13:20:41.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:41.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836684 -lt 21474836686 2026-02-19T13:20:41.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:42.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:42.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:42.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836687 -lt 21474836686 2026-02-19T13:20:42.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:42.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705868 2026-02-19T13:20:42.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:42.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:42.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705868 2026-02-19T13:20:42.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:42.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705868 2026-02-19T13:20:42.425 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705868 2026-02-19T13:20:42.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705868' 2026-02-19T13:20:42.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:42.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705868 -lt 38654705868 2026-02-19T13:20:42.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:42.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575050 2026-02-19T13:20:42.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:42.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:42.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575050 2026-02-19T13:20:42.592 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:42.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575050 2026-02-19T13:20:42.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575050' 2026-02-19T13:20:42.594 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575050 2026-02-19T13:20:42.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:42.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575050 -lt 55834575050 2026-02-19T13:20:42.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:42.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:42.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:42.907 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:42.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:42.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:42.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:42.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:42.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:43.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:43.076 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:43.076 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:43.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:43.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:43.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:43.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836689 2026-02-19T13:20:43.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836689 2026-02-19T13:20:43.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836689' 2026-02-19T13:20:43.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:43.153 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:43.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705871 2026-02-19T13:20:43.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705871 2026-02-19T13:20:43.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836689 1-38654705871' 2026-02-19T13:20:43.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:43.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:43.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575053 2026-02-19T13:20:43.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575053 2026-02-19T13:20:43.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836689 1-38654705871 2-55834575053' 2026-02-19T13:20:43.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:43.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836689 2026-02-19T13:20:43.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:43.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:43.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836689 2026-02-19T13:20:43.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:43.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836689 2026-02-19T13:20:43.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836689' 2026-02-19T13:20:43.303 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836689 2026-02-19T13:20:43.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:43.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836690 -lt 21474836689 2026-02-19T13:20:43.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:43.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705871 2026-02-19T13:20:43.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:43.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:43.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705871 2026-02-19T13:20:43.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:43.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705871 2026-02-19T13:20:43.464 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705871 2026-02-19T13:20:43.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705871' 2026-02-19T13:20:43.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:43.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705871 -lt 38654705871 2026-02-19T13:20:43.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:43.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575053 2026-02-19T13:20:43.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:43.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:43.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575053 2026-02-19T13:20:43.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:43.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575053 2026-02-19T13:20:43.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575053' 2026-02-19T13:20:43.628 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575053 2026-02-19T13:20:43.628 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:43.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575052 -lt 55834575053 2026-02-19T13:20:43.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:44.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:44.796 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:44.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575052 -lt 55834575053 2026-02-19T13:20:44.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:45.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:20:45.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:46.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575055 -lt 55834575053 2026-02-19T13:20:46.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:46.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:46.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:46.264 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:46.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:20:46.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:20:46.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:20:46.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:46.274 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:46.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:46.430 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:20:46.430 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:20:46.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:46.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:46.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:46.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836694 2026-02-19T13:20:46.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836694 2026-02-19T13:20:46.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836694' 2026-02-19T13:20:46.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:46.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:46.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705875 2026-02-19T13:20:46.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705875 2026-02-19T13:20:46.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836694 1-38654705875' 2026-02-19T13:20:46.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:46.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:46.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834575057 2026-02-19T13:20:46.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834575057 2026-02-19T13:20:46.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836694 1-38654705875 2-55834575057' 2026-02-19T13:20:46.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:46.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836694 2026-02-19T13:20:46.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:46.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:46.654 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836694 2026-02-19T13:20:46.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:46.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836694 2026-02-19T13:20:46.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836694' 2026-02-19T13:20:46.657 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836694 2026-02-19T13:20:46.657 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:46.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836692 -lt 21474836694 2026-02-19T13:20:46.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:47.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:47.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:47.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836695 -lt 21474836694 2026-02-19T13:20:47.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:47.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705875 2026-02-19T13:20:47.973 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:47.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:47.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705875 2026-02-19T13:20:47.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:47.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705875 2026-02-19T13:20:47.976 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705875 2026-02-19T13:20:47.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705875' 2026-02-19T13:20:47.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:48.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705876 -lt 38654705875 2026-02-19T13:20:48.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:48.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834575057 2026-02-19T13:20:48.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:48.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:48.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834575057 2026-02-19T13:20:48.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:48.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834575057 2026-02-19T13:20:48.148 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 55834575057 2026-02-19T13:20:48.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834575057' 2026-02-19T13:20:48.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:48.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834575058 -lt 55834575057 2026-02-19T13:20:48.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:20:48.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:20:48.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:20:48.457 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:20:48.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: break 2026-02-19T13:20:48.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: set +o pipefail 2026-02-19T13:20:48.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:382: _scrub_abort: sleep 5 2026-02-19T13:20:53.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:384: _scrub_abort: grep 'nodeep_scrub set, aborting' td/osd-scrub-test/osd.1.log 2026-02-19T13:20:53.477 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:18:51.336+0000 7f96c1fa6640 10 osd.1 pg_epoch: 19 pg[1.0( v 17'1000 (0'0,17'1000] local-lis/les=14/15 n=1000 ec=14/14 lis/c=14/14 les/c/f=15/16/0 sis=14) [1,0,2] r=0 lpr=14 crt=17'1000 lcod 17'999 mlcod 17'999 active+clean+scrubbing+deep(periodic-regular)] scrubber: nodeep_scrub set, aborting 2026-02-19T13:20:53.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:390: _scrub_abort: get_last_scrub_stamp 1.0 2026-02-19T13:20:53.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:20:53.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:20:53.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:20:53.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:20:53.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:390: _scrub_abort: local last_scrub=2026-02-19T13:18:16.573991+0000 2026-02-19T13:20:53.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:391: _scrub_abort: ceph config set osd osd_scrub_sleep 0.1 2026-02-19T13:20:53.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:393: _scrub_abort: ceph osd unset nodeep-scrub 2026-02-19T13:20:53.989 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is unset 2026-02-19T13:20:53.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:394: _scrub_abort: '[' deep-scrub = deep-scrub ']' 2026-02-19T13:20:53.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:396: _scrub_abort: ceph osd unset noscrub 2026-02-19T13:20:54.189 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is unset 2026-02-19T13:20:54.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:398: _scrub_abort: TIMEOUT=500 2026-02-19T13:20:54.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:399: _scrub_abort: wait_for_scrub 1.0 2026-02-19T13:18:16.573991+0000 2026-02-19T13:20:54.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T13:20:54.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:18:16.573991+0000 2026-02-19T13:20:54.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:20:54.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:20:54.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:20:54.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:20:54.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:20:54.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:20:54.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:20:54.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:20:54.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:20:54.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:20:55.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:20:55.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:20:55.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:20:55.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:20:55.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:20:55.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:20:55.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:20:55.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:20:55.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:20:56.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:20:56.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:20:56.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:20:56.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:20:56.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:20:56.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:20:56.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:20:56.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:20:56.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:20:57.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:20:57.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:20:57.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:20:57.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:20:57.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:20:57.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:20:57.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:20:57.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:20:57.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:20:58.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:20:58.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:20:58.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:20:58.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:20:58.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:20:58.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:20:58.848 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:20:59.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:20:59.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:00.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:00.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:00.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:00.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:00.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:00.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:00.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:00.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:00.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:01.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:01.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:01.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:01.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:01.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:01.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:01.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:01.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:01.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:02.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:02.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:02.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:02.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:02.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:02.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:02.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:02.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:02.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:03.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:03.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:03.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:03.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:03.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:03.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:03.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:03.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:03.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:04.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:04.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:04.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:04.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:04.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:04.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:04.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:04.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:04.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:05.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:05.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:05.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:05.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:05.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:05.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:05.822 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:05.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:05.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:06.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:06.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:06.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:06.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:06.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:06.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:06.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:07.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:07.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:08.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:08.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:08.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:08.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:08.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:08.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:08.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:08.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:08.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:09.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:09.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:09.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:09.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:09.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:09.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:09.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:09.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:09.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:10.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:10.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:10.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:10.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:10.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:10.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:10.454 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:10.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:10.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:11.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:11.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:11.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:11.612 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:11.613 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:11.613 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:11.613 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:11.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:11.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:12.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:12.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:12.771 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:12.771 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:12.771 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:12.771 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:12.771 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:12.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:12.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:13.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:13.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:13.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:13.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:13.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:13.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:13.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:14.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:14.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:15.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:15.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:15.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:15.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:15.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:15.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:15.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:15.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:15.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:16.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:16.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:16.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:16.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:16.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:16.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:16.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:16.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:16.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:17.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:17.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:17.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:17.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:17.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:17.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:17.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:17.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:17.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:18.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:18.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:18.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:18.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:18.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:18.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:18.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:18.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:18.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:19.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:19.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:19.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:19.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:19.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:19.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:19.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:19.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:19.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:20.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:20.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:20.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:20.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:20.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:20.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:20.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:21.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:21.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:22.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:22.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:22.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:22.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:22.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:22.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:22.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:22.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:22.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:23.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:23.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:23.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:23.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:23.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:23.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:23.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:23.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:23.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:24.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:24.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:24.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:24.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:24.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:24.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:24.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:24.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:24.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:25.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:25.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:25.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:25.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:25.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:25.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:25.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:25.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:25.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:26.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:26.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:26.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:26.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:26.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:26.673 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:26.673 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:26.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:26.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:27.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:27.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:27.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:27.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:27.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:27.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:27.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:27.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:27.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:28.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:28.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:28.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:28.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:28.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:28.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:28.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:29.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:29.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:30.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:30.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:30.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:30.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:30.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:30.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:30.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:30.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:30.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:31.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:31.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:31.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:31.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:31.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:31.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:31.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:31.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:31.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:32.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:32.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:32.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:32.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:32.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:32.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:32.471 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:32.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:32.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:33.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:33.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:33.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:33.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:33.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:33.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:33.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:33.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:33.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:34.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:34.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:34.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:34.790 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:34.791 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:34.791 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:34.791 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:34.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:34.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:35.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:35.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:35.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:35.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:35.950 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:35.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:35.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:36.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:36.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:37.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:37.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:37.109 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:37.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:37.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:37.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:37.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:37.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:37.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:38.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:38.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:38.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:38.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:38.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:38.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:38.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:38.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:38.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:39.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:39.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:39.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:39.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:39.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:39.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:39.425 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:39.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:39.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:40.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:40.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:40.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:40.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:40.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:40.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:40.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:40.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:40.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:41.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:41.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:41.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:41.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:41.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:41.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:41.736 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:41.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:41.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:42.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:42.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:42.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:42.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:42.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:42.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:42.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:43.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:43.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:44.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:44.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:44.049 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:44.049 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:44.049 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:44.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:44.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:44.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:44.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:45.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:45.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:45.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:45.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:45.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:45.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:45.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:45.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:45.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:46.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:46.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:46.371 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:46.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:46.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:46.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:46.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:46.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:46.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:47.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:47.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:47.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:47.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:47.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:47.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:47.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:47.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:47.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:48.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:48.694 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:48.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:48.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:48.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:48.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:48.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:48.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:48.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:49.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:49.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:49.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:49.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:49.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:49.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:49.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:50.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:50.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:51.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:51.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:51.008 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:51.008 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:51.008 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:51.008 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:51.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:51.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:51.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:52.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:52.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:52.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:52.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:52.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:52.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:52.162 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:52.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:52.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:53.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:53.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:53.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:53.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:53.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:53.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:53.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:53.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:53.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:54.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:54.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:54.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:54.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:54.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:54.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:54.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:54.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:54.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:55.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:55.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:55.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:55.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:55.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:55.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:55.641 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:55.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:55.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:56.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:56.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:56.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:56.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:56.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:56.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:56.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:56.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:56.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:57.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:57.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:57.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:57.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:57.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:57.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:57.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:58.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:58.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:21:59.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:21:59.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:21:59.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:21:59.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:21:59.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:21:59.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:21:59.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:21:59.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:21:59.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:00.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:00.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:00.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:00.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:00.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:00.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:00.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:00.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:00.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:01.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:01.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:01.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:01.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:01.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:01.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:01.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:01.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:01.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:02.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:02.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:02.597 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:02.597 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:02.597 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:02.598 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:02.598 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:02.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:02.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:03.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:03.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:03.758 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:03.758 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:03.758 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:03.759 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:03.759 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:03.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:03.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:04.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:04.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:04.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:04.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:04.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:04.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:04.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:05.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:05.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:06.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:06.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:06.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:06.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:06.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:06.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:06.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:06.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:06.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:07.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:07.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:07.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:07.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:07.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:07.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:07.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:08.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:08.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:08.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:08.385 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:08.385 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:08.385 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:08.385 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:08.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:08.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:09.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:09.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:09.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:09.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:09.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:09.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:09.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:09.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:09.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:10.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:10.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:10.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:10.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:10.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:10.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:10.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:10.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:10.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:11.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:11.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:11.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:11.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:11.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:11.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:11.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:12.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:12.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:13.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:13.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:13.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:13.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:13.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:13.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:13.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:13.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:13.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:14.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:14.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:14.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:14.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:14.193 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:14.193 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:14.193 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:14.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:14.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:15.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:15.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:15.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:15.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:15.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:15.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:15.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:15.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:15.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:16.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:16.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:16.513 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:16.513 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:16.513 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:16.513 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:16.513 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:16.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:16.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:17.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:17.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:17.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:17.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:17.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:17.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:17.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:17.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:17.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:18.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:18.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:18.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:18.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:18.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:18.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:18.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:18.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:18.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:19.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:19.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:19.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:19.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:19.991 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:19.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:19.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:20.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:20.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:21.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:21.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:21.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:21.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:21.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:21.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:21.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:21.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:21.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:22.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:22.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:22.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:22.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:22.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:22.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:22.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:22.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:22.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:23.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:23.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:23.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:23.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:23.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:23.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:23.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:23.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:23.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:24.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:24.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:24.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:24.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:24.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:24.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:24.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:24.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:24.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:25.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:25.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:25.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:25.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:25.781 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:25.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:25.782 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:25.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:25.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:26.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:26.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:26.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:26.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:26.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:26.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:26.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:27.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:27.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:28.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:28.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:28.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:28.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:28.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:28.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:28.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:28.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:28.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:29.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:29.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:29.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:29.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:29.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:29.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:29.252 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:29.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:29.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:30.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:30.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:30.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:30.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:30.410 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:30.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:30.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:30.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:30.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:31.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:31.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:31.571 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:31.571 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:31.571 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:31.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:31.572 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:31.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:31.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:32.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:32.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:32.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:32.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:32.734 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:32.734 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:32.734 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:32.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:32.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:33.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:33.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:33.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:33.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:33.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:33.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:33.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:34.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:34.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:35.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:35.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:35.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:35.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:35.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:35.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:35.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:35.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:35.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:36.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:36.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:36.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:36.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:36.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:36.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:36.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:36.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:36.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:37.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:37.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:37.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:37.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:37.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:37.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:37.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:37.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:37.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:38.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:38.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:38.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:38.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:38.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:38.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:38.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:38.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:38.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:39.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:39.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:39.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:39.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:39.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:39.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:39.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:39.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:39.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:40.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:40.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:40.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:40.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:40.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:40.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:40.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:40.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:40.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:41.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:41.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:41.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:41.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:41.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:41.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:41.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:42.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:42.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:43.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:43.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:43.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:43.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:43.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:43.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:43.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:43.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:43.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:44.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:44.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:44.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:44.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:44.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:44.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:44.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:44.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:44.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:45.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:45.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:45.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:45.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:45.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:45.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:45.463 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:45.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:45.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:46.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:46.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:46.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:46.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:46.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:46.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:46.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:46.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:46.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:47.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:47.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:47.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:47.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:47.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:47.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:47.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:47.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:47.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:48.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:48.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:48.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:48.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:48.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:48.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:48.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:49.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:49.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:50.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:50.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:50.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:50.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:50.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:50.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:50.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:50.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:50.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:51.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:51.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:51.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:51.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:51.249 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:51.249 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:51.249 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:51.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:51.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:52.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:52.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:52.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:52.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:52.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:52.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:52.403 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:52.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:52.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:53.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:53.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:53.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:53.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:53.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:53.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:53.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:53.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:53.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:54.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:54.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:54.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:54.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:54.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:54.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:54.713 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:54.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:54.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:55.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:55.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:55.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:55.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:55.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:55.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:55.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:56.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:56.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:57.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:57.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:57.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:57.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:57.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:57.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:57.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:57.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:57.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:58.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:58.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:58.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:58.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:58.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:58.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:58.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:58.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:58.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:22:59.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:22:59.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:22:59.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:22:59.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:22:59.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:22:59.341 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:22:59.341 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:22:59.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:22:59.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:00.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:00.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:00.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:00.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:00.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:00.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:00.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:00.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:00.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:01.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:01.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:01.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:01.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:01.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:01.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:01.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:01.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:01.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:02.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:02.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:02.820 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:02.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:02.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:02.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:02.821 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:02.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:02.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:03.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:03.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:03.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:03.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:03.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:03.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:03.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:04.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:04.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:05.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:05.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:05.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:05.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:05.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:05.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:05.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:05.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:05.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:06.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:06.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:06.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:06.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:06.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:06.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:06.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:06.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:06.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:07.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:07.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:07.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:07.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:07.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:07.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:07.458 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:07.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:07.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:08.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:08.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:08.609 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:08.609 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:08.609 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:08.610 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:08.610 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:08.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:08.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:09.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:09.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:09.764 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:09.764 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:09.764 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:09.764 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:09.764 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:09.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:09.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:10.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:10.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:10.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:10.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:10.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:10.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:10.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:11.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:11.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:12.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:12.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:12.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:12.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:12.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:12.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:12.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:12.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:12.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:13.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:13.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:13.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:13.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:13.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:13.236 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:13.236 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:13.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:13.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:14.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:14.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:14.393 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:14.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:14.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:14.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:14.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:14.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:14.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:15.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:15.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:15.552 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:15.552 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:15.552 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:15.552 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:15.552 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:15.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:15.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:16.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:16.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:16.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:16.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:16.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:16.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:16.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:16.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:16.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:17.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:17.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:17.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:17.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:17.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:17.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:17.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:18.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:18.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:19.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:19.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:19.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:19.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:19.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:19.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:19.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:19.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:19.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:20.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:20.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:20.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:20.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:20.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:20.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:20.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:20.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:20.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:21.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:21.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:21.355 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:21.355 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:21.355 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:21.355 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:21.355 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:21.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:21.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:22.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:22.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:22.514 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:22.514 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:22.514 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:22.514 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:22.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:22.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:22.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:23.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:23.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:23.673 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:23.673 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:23.673 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:23.673 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:23.673 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:23.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:23.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:24.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:24.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:24.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:24.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:24.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:24.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:24.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:24.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:24.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:25.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:25.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:25.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:25.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:25.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:25.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:25.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:26.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:26.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:27.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:27.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:27.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:27.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:27.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:27.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:27.155 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:27.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:27.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:28.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:28.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:28.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:28.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:28.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:28.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:28.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:28.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:28.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:29.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:29.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:29.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:29.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:29.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:29.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:29.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:29.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:29.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:30.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:30.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:30.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:30.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:30.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:30.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:30.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:30.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:30.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:31.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:31.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:31.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:31.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:31.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:31.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:31.792 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:31.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:31.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:32.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:32.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:32.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:32.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:32.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:32.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:32.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:33.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:33.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:34.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:34.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:34.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:34.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:34.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:34.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:34.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:34.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:34.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:35.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:35.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:35.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:35.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:35.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:35.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:35.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:35.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:35.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:36.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:36.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:36.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:36.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:36.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:36.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:36.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:36.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:36.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:37.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:37.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:37.598 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:37.598 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:37.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:37.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:37.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:37.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:37.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:38.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:38.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:38.756 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:38.756 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:38.756 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:38.756 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:38.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:38.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:38.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:39.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:39.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:39.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:39.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:39.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:39.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:39.918 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:40.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:40.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:41.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:41.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:41.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:41.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:41.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:41.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:41.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:41.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:41.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:42.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:42.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:42.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:42.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:42.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:42.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:42.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:42.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:42.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:43.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:43.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:43.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:43.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:43.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:43.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:43.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:43.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:43.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:44.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:44.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:44.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:44.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:44.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:44.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:44.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:44.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:44.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:45.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:45.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:45.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:45.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:45.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:45.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:45.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:45.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:45.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:46.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:46.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:46.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:46.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:46.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:46.862 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:46.862 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:47.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:47.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:48.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:48.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:48.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:48.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:48.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:48.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:48.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:48.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:48.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:49.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:49.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:49.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:49.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:49.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:49.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:49.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:49.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:49.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:50.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:50.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:50.339 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:50.339 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:50.339 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:50.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:50.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:50.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:50.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:51.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:51.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:51.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:51.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:51.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:51.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:51.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:51.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:51.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:52.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:52.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:52.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:52.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:52.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:52.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:52.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:52.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:52.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:53.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:53.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:53.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:53.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:53.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:53.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:53.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:53.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:53.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:54.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:54.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:54.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:54.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:54.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:54.974 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:54.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:55.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:55.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:56.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:56.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:56.133 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:56.133 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:56.133 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:56.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:56.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:56.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:56.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:57.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:57.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:57.293 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:57.293 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:57.293 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:57.294 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:57.294 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:57.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:57.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:58.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:58.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:58.454 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:58.454 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:58.454 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:58.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:58.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:58.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:58.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:23:59.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:23:59.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:23:59.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:23:59.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:23:59.614 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:23:59.615 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:23:59.615 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:23:59.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:23:59.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:00.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:00.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:00.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:00.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:00.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:00.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:00.773 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:00.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:00.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:01.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:01.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:01.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:01.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:01.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:01.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:01.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:02.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:02.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:03.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:03.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:03.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:03.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:03.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:03.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:03.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:03.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:03.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:04.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:04.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:04.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:04.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:04.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:04.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:04.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:04.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:05.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:05.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:05.406 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:05.406 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:05.406 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:05.406 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:05.406 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:05.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:05.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:06.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:06.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:06.564 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:06.564 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:06.564 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:06.564 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:06.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:06.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:06.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:07.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:07.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:07.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:07.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:07.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:07.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:07.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:07.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:07.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:08.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:08.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:08.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:08.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:08.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:08.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:08.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:09.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:09.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:10.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:10.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:10.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:10.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:10.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:10.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:10.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:10.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:10.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:11.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:11.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:11.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:11.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:11.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:11.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:11.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:11.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:11.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:12.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:12.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:12.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:12.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:12.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:12.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:12.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:12.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:12.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:13.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:13.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:13.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:13.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:13.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:13.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:13.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:13.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:13.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:14.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:14.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:14.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:14.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:14.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:14.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:14.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:14.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:14.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:15.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:15.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:15.827 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:15.827 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:15.827 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:15.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:15.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:15.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:15.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:16.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:16.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:16.985 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:16.985 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:16.985 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:16.985 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:16.985 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:17.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:17.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:18.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:18.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:18.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:18.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:18.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:18.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:18.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:18.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:18.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:19.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:19.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:19.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:19.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:19.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:19.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:19.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:19.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:19.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:20.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:20.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:20.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:20.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:20.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:20.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:20.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:20.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:20.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:21.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:21.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:21.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:21.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:21.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:21.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:21.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:21.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:21.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:22.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:22.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:22.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:22.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:22.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:22.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:22.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:22.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:22.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:23.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:23.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:23.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:23.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:23.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:23.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:23.939 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:24.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:24.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:25.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:25.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:25.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:25.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:25.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:25.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:25.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:25.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:25.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:26.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:26.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:26.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:26.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:26.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:26.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:26.254 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:26.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:26.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:27.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:27.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:27.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:27.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:27.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:27.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:27.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:27.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:27.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:28.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:28.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:28.565 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:28.566 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:28.566 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:28.566 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:28.566 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:28.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:28.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:29.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:29.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:29.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:29.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:29.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:29.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:29.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:29.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:29.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:30.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:30.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:30.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:30.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:30.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:30.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:30.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:31.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:31.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:32.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:32.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:32.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:32.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:32.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:32.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:32.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:32.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:32.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:33.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:33.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:33.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:33.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:33.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:33.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:33.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:33.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:33.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:34.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:34.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:34.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:34.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:34.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:34.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:34.363 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:34.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:34.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:35.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:35.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:35.517 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:35.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:35.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:35.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:35.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:35.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:35.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:36.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:36.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:36.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:36.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:36.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:36.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:36.677 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:36.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:36.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:37.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:37.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:37.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:37.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:37.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:37.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:37.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:37.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:37.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:38.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:38.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:38.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:38.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:38.996 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:38.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:38.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:39.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:39.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:40.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:40.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:40.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:40.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:40.157 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:40.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:40.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:40.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:40.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:41.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:41.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:41.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:41.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:41.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:41.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:41.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:41.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:41.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:42.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:42.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:42.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:42.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:42.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:42.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:42.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:42.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:42.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:43.627 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:43.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:43.628 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:43.628 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:43.628 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:43.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:43.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:43.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:43.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:44.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:44.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:44.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:44.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:44.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:44.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:44.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:44.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:44.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:45.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:45.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:45.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:45.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:45.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:45.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:45.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:46.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:46.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:47.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:47.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:47.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:47.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:47.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:47.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:47.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:47.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:47.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:48.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:48.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:48.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:48.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:48.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:48.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:48.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:48.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:48.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:49.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:49.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:49.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:49.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:49.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:49.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:49.423 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:49.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:49.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:50.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:50.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:50.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:50.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:50.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:50.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:50.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:50.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:50.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:51.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:51.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:51.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:51.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:51.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:51.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:51.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:51.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:51.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:52.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:52.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:52.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:52.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:52.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:52.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:52.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:53.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:53.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:54.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:54.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:54.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:54.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:54.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:54.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:54.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:54.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:54.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:55.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:55.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:55.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:55.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:55.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:55.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:55.216 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:55.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:55.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:56.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:56.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:56.371 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:56.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:56.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:56.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:56.372 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:56.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:56.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:57.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:57.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:57.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:57.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:57.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:57.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:57.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:57.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:57.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:58.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:58.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:58.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:58.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:58.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:58.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:58.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:58.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:58.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:24:59.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:24:59.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:24:59.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:24:59.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:24:59.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:24:59.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:24:59.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:24:59.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:24:59.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:00.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:00.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:00.994 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:00.995 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:00.995 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:00.995 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:00.995 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:01.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:01.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:02.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:02.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:02.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:02.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:02.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:02.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:02.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:02.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:02.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:03.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:03.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:03.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:03.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:03.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:03.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:03.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:03.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:03.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:04.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:04.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:04.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:04.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:04.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:04.463 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:04.463 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:04.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:04.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:05.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:05.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:05.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:05.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:05.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:05.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:05.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:05.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:05.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:06.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:06.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:06.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:06.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:06.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:06.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:06.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:06.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:06.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:07.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:07.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:07.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:07.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:07.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:07.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:07.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:08.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:08.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:09.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:09.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:09.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:09.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:09.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:09.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:09.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:09.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:09.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:10.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:10.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:10.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:10.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:10.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:10.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:10.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:10.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:10.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:11.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:11.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:11.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:11.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:11.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:11.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:11.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:11.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:11.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:12.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:12.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:12.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:12.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:12.564 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:12.564 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:12.564 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:12.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:12.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:13.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:13.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:13.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:13.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:13.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:13.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:13.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:13.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:13.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:14.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:14.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:14.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:14.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:14.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:14.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:14.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:15.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:15.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:16.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:16.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:16.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:16.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:16.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:16.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:16.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:16.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:16.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:17.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:17.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:17.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:17.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:17.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:17.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:17.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:17.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:17.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:18.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:18.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:18.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:18.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:18.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:18.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:18.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:18.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:18.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:19.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:19.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:19.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:19.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:19.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:19.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:19.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:19.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:19.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:20.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:20.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:20.664 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:20.664 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:20.664 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:20.665 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:20.665 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:20.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:20.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:21.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:21.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:21.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:21.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:21.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:21.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:21.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:21.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:21.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:22.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:22.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:22.985 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:22.985 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:22.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:22.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:22.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:23.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:23.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:24.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:24.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:24.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:24.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:24.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:24.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:24.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:24.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:24.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:25.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:25.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:25.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:25.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:25.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:25.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:25.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:25.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:25.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:26.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:26.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:26.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:26.466 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:26.466 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:26.466 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:26.466 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:26.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:26.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:27.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:27.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:27.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:27.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:27.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:27.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:27.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:27.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:27.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:28.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:28.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:28.787 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:28.787 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:28.788 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:28.788 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:28.788 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:28.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:28.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:29.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:29.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:29.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:29.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:29.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:29.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:29.951 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:30.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:30.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:31.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:31.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:31.107 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:31.107 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:31.107 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:31.107 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:31.107 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:31.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:31.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:32.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:32.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:32.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:32.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:32.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:32.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:32.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:32.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:32.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:33.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:33.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:33.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:33.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:33.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:33.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:33.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:33.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:33.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:34.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:34.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:34.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:34.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:34.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:34.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:34.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:34.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:34.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:35.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:35.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:35.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:35.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:35.739 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:35.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:35.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:35.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:35.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:36.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:36.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:36.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:36.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:36.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:36.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:36.894 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:37.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:37.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:38.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:38.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:38.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:38.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:38.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:38.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:38.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:38.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:38.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:39.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:39.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:39.205 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:39.205 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:39.205 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:39.205 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:39.205 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:39.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:39.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:40.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:40.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:40.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:40.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:40.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:40.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:40.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:40.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:40.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:41.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:41.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:41.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:41.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:41.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:41.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:41.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:41.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:41.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:42.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:42.693 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:42.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:42.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:42.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:42.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:42.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:42.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:42.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:43.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:43.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:43.849 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:43.849 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:43.849 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:43.849 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:43.849 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:44.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:44.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:45.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:45.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:45.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:45.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:45.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:45.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:45.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:45.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:45.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:46.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:46.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:46.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:46.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:46.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:46.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:46.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:46.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:46.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:47.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:47.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:47.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:47.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:47.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:47.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:47.319 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:47.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:47.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:48.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:48.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:48.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:48.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:48.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:48.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:48.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:48.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:48.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:49.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:49.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:49.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:49.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:49.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:49.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:49.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:49.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:49.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:50.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:50.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:50.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:50.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:50.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:50.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:50.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:50.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:50.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:51.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:51.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:51.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:51.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:51.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:51.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:51.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:52.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:52.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:53.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:53.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:53.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:53.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:53.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:53.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:53.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:53.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:53.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:54.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:54.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:54.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:54.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:54.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:54.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:54.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:54.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:54.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:55.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:55.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:55.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:55.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:55.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:55.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:55.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:55.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:55.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:56.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:56.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:56.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:56.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:56.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:56.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:56.585 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:56.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:56.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:57.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:57.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:57.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:57.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:57.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:57.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:57.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:57.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:57.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:25:58.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:25:58.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:25:58.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:25:58.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:25:58.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:25:58.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:25:58.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:25:59.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:25:59.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:00.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:00.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:00.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:00.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:00.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:00.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:00.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:00.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:00.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:01.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:01.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:01.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:01.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:01.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:01.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:01.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:01.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:01.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:02.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:02.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:02.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:02.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:02.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:02.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:02.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:02.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:02.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:03.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:03.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:03.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:03.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:03.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:03.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:03.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:03.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:03.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:04.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:04.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:04.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:04.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:04.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:04.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:04.688 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:04.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:04.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:05.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:05.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:05.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:05.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:05.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:05.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:05.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:05.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:05.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:06.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:06.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:06.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:06.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:06.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:06.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:06.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:07.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:07.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:08.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:08.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:08.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:08.160 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:08.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:08.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:08.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:08.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:08.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:09.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:09.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:09.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:09.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:09.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:09.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:09.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:09.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:09.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:10.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:10.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:10.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:10.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:10.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:10.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:10.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:10.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:10.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:11.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:11.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:11.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:11.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:11.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:11.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:11.642 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:11.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:11.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:12.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:12.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:12.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:12.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:12.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:12.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:12.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:12.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:12.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:13.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:13.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:13.962 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:13.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:13.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:13.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:13.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:14.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:14.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:15.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:15.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:15.122 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:15.123 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:15.123 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:15.123 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:15.123 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:15.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:15.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:16.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:16.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:16.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:16.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:16.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:16.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:16.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:16.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:16.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:17.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:17.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:17.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:17.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:17.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:17.443 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:17.443 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:17.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:17.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:18.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:18.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:18.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:18.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:18.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:18.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:18.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:18.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:18.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:19.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:19.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:19.756 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:19.756 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:19.756 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:19.756 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:19.756 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:19.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:19.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:20.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:20.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:20.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:20.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:20.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:20.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:20.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:21.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:21.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:22.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:22.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:22.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:22.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:22.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:22.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:22.070 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:22.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:22.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:23.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:23.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:23.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:23.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:23.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:23.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:23.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:23.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:23.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:24.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:24.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:24.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:24.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:24.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:24.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:24.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:24.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:24.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:25.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:25.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:25.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:25.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:25.548 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:25.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:25.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:25.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:25.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:26.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:26.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:26.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:26.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:26.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:26.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:26.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:26.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:26.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:27.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:27.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:27.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:27.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:27.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:27.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:27.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:28.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:28.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:29.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:29.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:29.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:29.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:29.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:29.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:29.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:29.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:29.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:30.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:30.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:30.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:30.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:30.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:30.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:30.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:30.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:30.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:31.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:31.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:31.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:31.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:31.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:31.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:31.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:31.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:31.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:32.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:32.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:32.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:32.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:32.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:32.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:32.491 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:32.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:32.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:33.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:33.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:33.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:33.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:33.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:33.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:33.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:33.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:33.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:34.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:34.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:34.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:34.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:34.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:34.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:34.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:34.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:34.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:35.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:35.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:35.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:35.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:35.957 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:35.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:35.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:36.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:36.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:37.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:37.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:37.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:37.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:37.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:37.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:37.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:37.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:37.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:38.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:38.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:38.274 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:38.274 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:38.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:38.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:38.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:38.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:38.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:39.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:39.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:39.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:39.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:39.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:39.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:39.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:39.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:39.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:40.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:40.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:40.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:40.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:40.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:40.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:40.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:40.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:40.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:41.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:41.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:41.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:41.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:41.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:41.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:41.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:41.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:41.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:42.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:42.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:42.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:42.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:42.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:42.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:42.896 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:43.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:43.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:44.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:44.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:44.055 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:44.055 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:44.055 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:44.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:44.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:44.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:44.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:45.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:45.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:45.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:45.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:45.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:45.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:45.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:45.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:45.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:46.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:46.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:46.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:46.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:46.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:46.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:46.374 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:46.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:46.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:47.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:47.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:47.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:47.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:47.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:47.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:47.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:47.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:47.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:48.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:48.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:48.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:48.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:48.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:48.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:48.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:48.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:48.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:49.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:49.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:49.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:49.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:49.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:49.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:49.847 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:50.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:50.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:51.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:51.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:51.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:51.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:51.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:51.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:51.005 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:51.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:51.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:52.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:52.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:52.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:52.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:52.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:52.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:52.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:52.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:52.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:53.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:53.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:53.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:53.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:53.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:53.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:53.328 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:53.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:53.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:54.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:54.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:54.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:54.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:54.487 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:54.487 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:54.487 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:54.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:54.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:55.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:55.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:55.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:55.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:55.645 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:55.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:55.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:55.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:55.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:56.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:56.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:56.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:56.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:56.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:56.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:56.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:56.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:56.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:57.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:57.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:57.961 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:57.961 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:57.961 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:57.961 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:57.961 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:58.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:58.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:26:59.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:26:59.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:26:59.122 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:26:59.122 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:26:59.122 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:26:59.123 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:26:59.123 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:26:59.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:26:59.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:27:00.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:27:00.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:27:00.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:27:00.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:27:00.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:27:00.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:27:00.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:27:00.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:18:16.573991+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:27:00.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:27:01.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:27:01.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:27:01.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:27:01.438 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:27:01.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:27:01.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:27:01.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:27:01.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:26:58.723983+0000 '>' 2026-02-19T13:18:16.573991+0000 2026-02-19T13:27:01.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:27:01.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:400: _scrub_abort: perf_counters td/osd-scrub-test 3 2026-02-19T13:27:01.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-19T13:27:01.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-19T13:27:01.597 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-19T13:27:01.598 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-19T13:27:01.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:27:01.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-19T13:27:01.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:27:01.678 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:27:01.678 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:27:01.678 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:01.678 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:01.678 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:27:01.678 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:27:01.678 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:01.679 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:01.679 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:01.679 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:01.679 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:01.679 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:01.679 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:01.679 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.679 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.679 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:01.679 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:27:01.679 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:01.679 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:01.680 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:27:01.680 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:27:01.680 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:01.680 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:01.680 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:01.680 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:01.680 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:01.680 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:01.680 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:01.680 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.680 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.680 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:01.681 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:27:01.681 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:01.681 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:01.681 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:27:01.681 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:27:01.681 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:01.681 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:01.681 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:01.681 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:01.681 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:01.681 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:01.681 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:01.681 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.682 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.682 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:01.682 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:27:01.682 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:01.682 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:01.682 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:27:01.682 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:27:01.682 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:01.682 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:01.682 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:01.682 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:01.682 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:01.683 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:01.683 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:01.683 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.683 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.683 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:27:01.683 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:01.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:27:01.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-19T13:27:01.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:27:01.749 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:27:01.749 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:27:01.749 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:01.749 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:01.749 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:01.750 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 67, 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:01.751 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.752 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:27:01.753 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:01.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:27:01.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-19T13:27:01.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:27:01.824 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:27:01.824 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:27:01.825 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:01.825 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:01.825 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:27:01.825 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:27:01.825 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:01.825 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:01.825 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:01.825 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:01.825 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:01.825 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:01.825 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:01.826 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.826 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.826 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:01.826 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:27:01.826 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:01.826 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:01.826 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:27:01.826 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:27:01.826 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:01.826 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:01.826 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:01.826 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:01.826 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:01.827 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:01.827 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:01.827 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.827 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.827 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:01.827 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:27:01.827 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:01.827 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:01.827 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:27:01.827 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:27:01.827 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:01.827 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:01.827 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:01.828 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:01.828 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:01.828 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:01.828 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:01.828 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.828 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.828 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:01.828 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:27:01.828 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:01.828 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:01.828 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:27:01.828 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:27:01.829 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:01.829 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:01.829 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:01.829 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:01.829 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:01.829 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:01.829 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:01.829 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.829 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:01.829 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:27:01.829 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:01.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_deep_scrub_abort ------------------' 2026-02-19T13:27:01.830 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Teardown Test TEST_deep_scrub_abort ------------------ 2026-02-19T13:27:01.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-19T13:27:01.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:27:01.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:27:01.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:27:01.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:27:01.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:27:01.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:27:01.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:27:01.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:27:01.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:27:01.934 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:27:01.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:27:01.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:27:01.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:27:01.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:27:01.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:27:01.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:27:01.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:27:01.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:27:01.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:27:01.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:27:01.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:27:01.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:27:02.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:27:02.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:02.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:02.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:27:02.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:27:02.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:27:02.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_deep_scrub_abort ------------------' 2026-02-19T13:27:02.004 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Complete Test TEST_deep_scrub_abort ------------------ 2026-02-19T13:27:02.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-19T13:27:02.004 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Prepare Test TEST_dump_scrub_schedule ------------------- 2026-02-19T13:27:02.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_dump_scrub_schedule -------------------' 2026-02-19T13:27:02.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-19T13:27:02.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-test 2026-02-19T13:27:02.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-test 2026-02-19T13:27:02.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:27:02.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:27:02.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:27:02.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:27:02.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:27:02.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:27:02.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:27:02.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:27:02.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:27:02.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:27:02.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:27:02.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:27:02.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:27:02.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:27:02.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:27:02.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:27:02.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:27:02.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:27:02.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:27:02.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:27:02.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:27:02.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:27:02.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:27:02.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:02.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:02.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:27:02.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:27:02.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:27:02.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-test 2026-02-19T13:27:02.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:27:02.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:02.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:02.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.511299 2026-02-19T13:27:02.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:27:02.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:27:02.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:27:02.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-19T13:27:02.022 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Run Test TEST_dump_scrub_schedule ----------------------- 2026-02-19T13:27:02.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_dump_scrub_schedule -----------------------' 2026-02-19T13:27:02.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_dump_scrub_schedule td/osd-scrub-test 2026-02-19T13:27:02.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:530: TEST_dump_scrub_schedule: local dir=td/osd-scrub-test 2026-02-19T13:27:02.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:531: TEST_dump_scrub_schedule: local poolname=test 2026-02-19T13:27:02.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:532: TEST_dump_scrub_schedule: local OSDS=3 2026-02-19T13:27:02.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:533: TEST_dump_scrub_schedule: local objects=90 2026-02-19T13:27:02.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:535: TEST_dump_scrub_schedule: TESTDATA=testdata.511299 2026-02-19T13:27:02.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:537: TEST_dump_scrub_schedule: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-19T13:27:02.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-test 2026-02-19T13:27:02.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:27:02.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:27:02.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:27:02.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-test/a 2026-02-19T13:27:02.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-19T13:27:02.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:27:02.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:27:02.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:27:02.043 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:27:02.044 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:02.044 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:02.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:27:02.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-19T13:27:02.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:27:02.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:27:02.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:27:02.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:27:02.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:27:02.076 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:27:02.076 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:27:02.076 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:27:02.077 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:27:02.077 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:27:02.077 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:02.077 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:02.077 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-mon.a.asok 2026-02-19T13:27:02.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:27:02.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.511299/ceph-mon.a.asok config get fsid 2026-02-19T13:27:02.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:27:02.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:27:02.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:27:02.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:27:02.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:27:02.135 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:27:02.135 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:27:02.136 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:27:02.136 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:27:02.136 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:02.136 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:02.136 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-mon.a.asok 2026-02-19T13:27:02.136 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:27:02.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.511299/ceph-mon.a.asok config get mon_host 2026-02-19T13:27:02.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:538: TEST_dump_scrub_schedule: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-19T13:27:02.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-test 2026-02-19T13:27:02.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:27:02.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:27:02.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:27:02.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-test/x 2026-02-19T13:27:02.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:27:02.279 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:27:02.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:27:02.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:27:02.280 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:27:02.280 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:02.280 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:02.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:27:02.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:27:02.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-19T13:27:02.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:552: TEST_dump_scrub_schedule: local 'ceph_osd_args=--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=1 --osd_scrub_retry_pg_state=2 --osd_scrub_retry_delay=2 --osd_scrub_sleep=0.2' 2026-02-19T13:27:02.300 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:554: TEST_dump_scrub_schedule: expr 3 - 1 2026-02-19T13:27:02.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:554: TEST_dump_scrub_schedule: seq 0 2 2026-02-19T13:27:02.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:554: TEST_dump_scrub_schedule: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:27:02.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:556: TEST_dump_scrub_schedule: run_osd td/osd-scrub-test 0 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=1 --osd_scrub_retry_pg_state=2 --osd_scrub_retry_delay=2 --osd_scrub_sleep=0.2 2026-02-19T13:27:02.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:27:02.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:27:02.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:27:02.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:27:02.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-19T13:27:02.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:27:02.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:27:02.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:27:02.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:27:02.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-19T13:27:02.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-19T13:27:02.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:27:02.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:27:02.306 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:27:02.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:27:02.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:27:02.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:27:02.307 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:27:02.307 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:02.307 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:02.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:27:02.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:27:02.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:27:02.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:27:02.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:27:02.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:27:02.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:27:02.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:27:02.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:27:02.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:27:02.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:27:02.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:27:02.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=1 --osd_scrub_retry_pg_state=2 --osd_scrub_retry_delay=2 --osd_scrub_sleep=0.2' 2026-02-19T13:27:02.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-19T13:27:02.308 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:27:02.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3ec600d8-3421-4a7c-862e-38bd99a01746 2026-02-19T13:27:02.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 3ec600d8-3421-4a7c-862e-38bd99a01746' 2026-02-19T13:27:02.309 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 3ec600d8-3421-4a7c-862e-38bd99a01746 2026-02-19T13:27:02.309 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:27:02.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCmD5dp/cbTEhAAsqaeobiZj92CawAtzr5EQQ== 2026-02-19T13:27:02.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCmD5dp/cbTEhAAsqaeobiZj92CawAtzr5EQQ=="}' 2026-02-19T13:27:02.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3ec600d8-3421-4a7c-862e-38bd99a01746 -i td/osd-scrub-test/0/new.json 2026-02-19T13:27:02.403 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:27:02.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-19T13:27:02.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=1 --osd_scrub_retry_pg_state=2 --osd_scrub_retry_delay=2 --osd_scrub_sleep=0.2 --mkfs --key AQCmD5dp/cbTEhAAsqaeobiZj92CawAtzr5EQQ== --osd-uuid 3ec600d8-3421-4a7c-862e-38bd99a01746 2026-02-19T13:27:02.431 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:02.426+0000 7f8692f0ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:02.432 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:02.430+0000 7f8692f0ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:02.433 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:02.430+0000 7f8692f0ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:02.434 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:02.430+0000 7f8692f0ca40 -1 bdev(0x558ceee94c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:27:02.434 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:02.430+0000 7f8692f0ca40 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-19T13:27:03.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-19T13:27:03.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:27:03.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:27:03.179 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:27:03.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:27:03.376 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:27:03.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:27:03.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=1 --osd_scrub_retry_pg_state=2 --osd_scrub_retry_delay=2 --osd_scrub_sleep=0.2 2026-02-19T13:27:03.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:27:03.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:27:03.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:27:03.404 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:03.402+0000 7fe348821a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:03.406 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:03.402+0000 7fe348821a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:03.407 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:03.406+0000 7fe348821a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:03.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:27:03.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:27:03.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:27:03.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:27:03.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:27:03.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:27:03.538 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:27:03.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:27:03.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:27:03.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:27:03.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:27:03.801 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:03.798+0000 7fe348821a40 -1 Falling back to public interface 2026-02-19T13:27:04.409 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:04.406+0000 7fe348821a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:27:04.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:27:04.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:27:04.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:27:04.697 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:27:04.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:27:04.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:27:04.853 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3399540747,v1:127.0.0.1:6803/3399540747] [v2:127.0.0.1:6804/3399540747,v1:127.0.0.1:6805/3399540747] exists,up 3ec600d8-3421-4a7c-862e-38bd99a01746 2026-02-19T13:27:04.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:27:04.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:27:04.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:27:04.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:554: TEST_dump_scrub_schedule: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:27:04.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:556: TEST_dump_scrub_schedule: run_osd td/osd-scrub-test 1 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=1 --osd_scrub_retry_pg_state=2 --osd_scrub_retry_delay=2 --osd_scrub_sleep=0.2 2026-02-19T13:27:04.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:27:04.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:27:04.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:27:04.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:27:04.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-19T13:27:04.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:27:04.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:27:04.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:27:04.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:27:04.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-19T13:27:04.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-19T13:27:04.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:27:04.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:27:04.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:27:04.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:27:04.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=1 --osd_scrub_retry_pg_state=2 --osd_scrub_retry_delay=2 --osd_scrub_sleep=0.2' 2026-02-19T13:27:04.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-19T13:27:04.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:27:04.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d586a89a-def6-4000-8726-309f2c091d36 2026-02-19T13:27:04.858 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 d586a89a-def6-4000-8726-309f2c091d36 2026-02-19T13:27:04.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 d586a89a-def6-4000-8726-309f2c091d36' 2026-02-19T13:27:04.858 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:27:04.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCoD5dp8MbZMxAAEAoOkYz760g+Nl0LFUQzIQ== 2026-02-19T13:27:04.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCoD5dp8MbZMxAAEAoOkYz760g+Nl0LFUQzIQ=="}' 2026-02-19T13:27:04.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d586a89a-def6-4000-8726-309f2c091d36 -i td/osd-scrub-test/1/new.json 2026-02-19T13:27:05.032 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:27:05.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-19T13:27:05.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=1 --osd_scrub_retry_pg_state=2 --osd_scrub_retry_delay=2 --osd_scrub_sleep=0.2 --mkfs --key AQCoD5dp8MbZMxAAEAoOkYz760g+Nl0LFUQzIQ== --osd-uuid d586a89a-def6-4000-8726-309f2c091d36 2026-02-19T13:27:05.062 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:05.058+0000 7f10980e2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:05.064 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:05.062+0000 7f10980e2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:05.064 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:05.062+0000 7f10980e2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:05.065 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:05.062+0000 7f10980e2a40 -1 bdev(0x55ea6cc54c00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:27:05.065 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:05.062+0000 7f10980e2a40 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-19T13:27:05.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-19T13:27:05.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:27:05.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:27:05.696 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:27:05.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:27:05.896 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:27:05.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:27:05.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=1 --osd_scrub_retry_pg_state=2 --osd_scrub_retry_delay=2 --osd_scrub_sleep=0.2 2026-02-19T13:27:05.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:27:05.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:27:05.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:27:05.924 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:05.922+0000 7ff33613fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:05.926 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:05.922+0000 7ff33613fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:05.927 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:05.926+0000 7ff33613fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:06.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:27:06.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:27:06.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:27:06.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:27:06.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:27:06.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:27:06.059 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:27:06.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:27:06.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:27:06.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:27:06.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:27:06.293 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:06.290+0000 7ff33613fa40 -1 Falling back to public interface 2026-02-19T13:27:06.876 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:06.874+0000 7ff33613fa40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:27:07.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:27:07.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:27:07.218 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:27:07.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:27:07.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:27:07.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:27:07.379 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3484122705,v1:127.0.0.1:6811/3484122705] [v2:127.0.0.1:6812/3484122705,v1:127.0.0.1:6813/3484122705] exists,up d586a89a-def6-4000-8726-309f2c091d36 2026-02-19T13:27:07.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:27:07.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:27:07.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:27:07.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:554: TEST_dump_scrub_schedule: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:27:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:556: TEST_dump_scrub_schedule: run_osd td/osd-scrub-test 2 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=1 --osd_scrub_retry_pg_state=2 --osd_scrub_retry_delay=2 --osd_scrub_sleep=0.2 2026-02-19T13:27:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:27:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:27:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:27:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:27:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-19T13:27:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:27:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:27:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:27:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:27:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-19T13:27:07.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-19T13:27:07.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:27:07.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:27:07.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:27:07.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:27:07.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:27:07.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:27:07.384 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:27:07.384 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:07.384 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:07.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:27:07.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:27:07.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:27:07.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:27:07.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:27:07.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:27:07.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:27:07.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:27:07.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:27:07.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:27:07.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:27:07.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:27:07.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=1 --osd_scrub_retry_pg_state=2 --osd_scrub_retry_delay=2 --osd_scrub_sleep=0.2' 2026-02-19T13:27:07.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-19T13:27:07.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:27:07.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=31f77fff-5eaa-434a-a393-124192643e25 2026-02-19T13:27:07.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 31f77fff-5eaa-434a-a393-124192643e25' 2026-02-19T13:27:07.387 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 31f77fff-5eaa-434a-a393-124192643e25 2026-02-19T13:27:07.387 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:27:07.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCrD5dptQCcFxAABb5hs7lUV6NFALlqPcfHMg== 2026-02-19T13:27:07.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCrD5dptQCcFxAABb5hs7lUV6NFALlqPcfHMg=="}' 2026-02-19T13:27:07.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 31f77fff-5eaa-434a-a393-124192643e25 -i td/osd-scrub-test/2/new.json 2026-02-19T13:27:07.546 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:27:07.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-19T13:27:07.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=1 --osd_scrub_retry_pg_state=2 --osd_scrub_retry_delay=2 --osd_scrub_sleep=0.2 --mkfs --key AQCrD5dptQCcFxAABb5hs7lUV6NFALlqPcfHMg== --osd-uuid 31f77fff-5eaa-434a-a393-124192643e25 2026-02-19T13:27:07.583 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:07.582+0000 7fc4fd0b4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:07.585 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:07.582+0000 7fc4fd0b4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:07.586 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:07.582+0000 7fc4fd0b4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:07.587 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:07.582+0000 7fc4fd0b4a40 -1 bdev(0x55f181778c00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:27:07.587 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:07.582+0000 7fc4fd0b4a40 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-19T13:27:08.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-19T13:27:08.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:27:08.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:27:08.268 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T13:27:08.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:27:08.469 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T13:27:08.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:27:08.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_retry_after_noscrub=1 --osd_scrub_retry_pg_state=2 --osd_scrub_retry_delay=2 --osd_scrub_sleep=0.2 2026-02-19T13:27:08.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:27:08.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:27:08.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:27:08.497 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:08.494+0000 7fe6e8976a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:08.499 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:08.498+0000 7fe6e8976a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:08.500 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:08.498+0000 7fe6e8976a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:08.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:27:08.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:27:08.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:27:08.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:27:08.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:27:08.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:27:08.631 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:27:08.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:27:08.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:27:08.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:27:08.788 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:27:08.941 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:08.938+0000 7fe6e8976a40 -1 Falling back to public interface 2026-02-19T13:27:09.573 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:09.570+0000 7fe6e8976a40 -1 osd.2 0 log_to_monitors true 2026-02-19T13:27:09.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:27:09.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:27:09.790 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:27:09.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:27:09.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:27:09.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:27:09.945 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1083917166,v1:127.0.0.1:6819/1083917166] [v2:127.0.0.1:6820/1083917166,v1:127.0.0.1:6821/1083917166] exists,up 31f77fff-5eaa-434a-a393-124192643e25 2026-02-19T13:27:09.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:27:09.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:27:09.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:27:09.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:560: TEST_dump_scrub_schedule: create_pool test 1 1 2026-02-19T13:27:09.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-19T13:27:10.135 INFO:tasks.workunit.client.0.trial144.stderr:pool 'test' created 2026-02-19T13:27:10.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:27:11.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:561: TEST_dump_scrub_schedule: wait_for_clean 2026-02-19T13:27:11.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:27:11.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:27:11.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:27:11.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:27:11.151 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:27:11.151 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:27:11.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:27:11.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:27:11.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:27:11.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:27:11.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:27:11.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:27:11.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:27:11.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:27:11.251 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:27:11.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:27:11.412 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:27:11.413 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:27:11.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:27:11.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:27:11.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:27:11.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-19T13:27:11.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-19T13:27:11.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-19T13:27:11.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:27:11.485 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:27:11.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672965 2026-02-19T13:27:11.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672965 2026-02-19T13:27:11.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-42949672965' 2026-02-19T13:27:11.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:27:11.557 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:27:11.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509442 2026-02-19T13:27:11.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509442 2026-02-19T13:27:11.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-42949672965 2-64424509442' 2026-02-19T13:27:11.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:27:11.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-19T13:27:11.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:27:11.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:27:11.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-19T13:27:11.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:27:11.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-19T13:27:11.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-19T13:27:11.635 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836487 2026-02-19T13:27:11.635 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:27:11.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T13:27:11.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:27:12.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:27:12.803 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:27:12.963 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836489 -lt 21474836487 2026-02-19T13:27:12.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:27:12.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672965 2026-02-19T13:27:12.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:27:12.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:27:12.966 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672965 2026-02-19T13:27:12.966 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:27:12.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672965 2026-02-19T13:27:12.968 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672965 2026-02-19T13:27:12.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672965' 2026-02-19T13:27:12.968 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:27:13.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672966 -lt 42949672965 2026-02-19T13:27:13.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:27:13.126 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509442 2026-02-19T13:27:13.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:27:13.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:27:13.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509442 2026-02-19T13:27:13.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:27:13.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509442 2026-02-19T13:27:13.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509442' 2026-02-19T13:27:13.130 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 64424509442 2026-02-19T13:27:13.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:27:13.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509444 -lt 64424509442 2026-02-19T13:27:13.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:27:13.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:27:13.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:27:13.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:27:13.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:27:13.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:27:13.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:27:13.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:27:13.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:27:13.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:27:13.492 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:27:13.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:27:13.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:27:13.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:27:13.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:27:13.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:27:13.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:27:13.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:27:13.842 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: ceph osd dump 2026-02-19T13:27:13.842 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: grep '^pool.*['\'']test['\'']' 2026-02-19T13:27:13.842 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: awk '{ print $2 }' 2026-02-19T13:27:13.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: poolid=1 2026-02-19T13:27:13.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: dd if=/dev/urandom of=testdata.511299 bs=1032 count=1 2026-02-19T13:27:13.998 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:27:13.998 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:27:13.998 INFO:tasks.workunit.client.0.trial144.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.757e-05 s, 27.5 MB/s 2026-02-19T13:27:13.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: seq 1 90 2026-02-19T13:27:13.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:13.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj1 testdata.511299 2026-02-19T13:27:14.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj2 testdata.511299 2026-02-19T13:27:14.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj3 testdata.511299 2026-02-19T13:27:14.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj4 testdata.511299 2026-02-19T13:27:14.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj5 testdata.511299 2026-02-19T13:27:14.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj6 testdata.511299 2026-02-19T13:27:14.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj7 testdata.511299 2026-02-19T13:27:14.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj8 testdata.511299 2026-02-19T13:27:14.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj9 testdata.511299 2026-02-19T13:27:14.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj10 testdata.511299 2026-02-19T13:27:14.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj11 testdata.511299 2026-02-19T13:27:14.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj12 testdata.511299 2026-02-19T13:27:14.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj13 testdata.511299 2026-02-19T13:27:14.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj14 testdata.511299 2026-02-19T13:27:14.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj15 testdata.511299 2026-02-19T13:27:14.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj16 testdata.511299 2026-02-19T13:27:14.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj17 testdata.511299 2026-02-19T13:27:14.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj18 testdata.511299 2026-02-19T13:27:14.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj19 testdata.511299 2026-02-19T13:27:14.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj20 testdata.511299 2026-02-19T13:27:14.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj21 testdata.511299 2026-02-19T13:27:14.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj22 testdata.511299 2026-02-19T13:27:14.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj23 testdata.511299 2026-02-19T13:27:14.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj24 testdata.511299 2026-02-19T13:27:14.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj25 testdata.511299 2026-02-19T13:27:14.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj26 testdata.511299 2026-02-19T13:27:14.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj27 testdata.511299 2026-02-19T13:27:14.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj28 testdata.511299 2026-02-19T13:27:14.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj29 testdata.511299 2026-02-19T13:27:14.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj30 testdata.511299 2026-02-19T13:27:14.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj31 testdata.511299 2026-02-19T13:27:14.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj32 testdata.511299 2026-02-19T13:27:14.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj33 testdata.511299 2026-02-19T13:27:14.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj34 testdata.511299 2026-02-19T13:27:14.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj35 testdata.511299 2026-02-19T13:27:14.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj36 testdata.511299 2026-02-19T13:27:14.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj37 testdata.511299 2026-02-19T13:27:14.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj38 testdata.511299 2026-02-19T13:27:14.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj39 testdata.511299 2026-02-19T13:27:14.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj40 testdata.511299 2026-02-19T13:27:14.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj41 testdata.511299 2026-02-19T13:27:14.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj42 testdata.511299 2026-02-19T13:27:14.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj43 testdata.511299 2026-02-19T13:27:14.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj44 testdata.511299 2026-02-19T13:27:14.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj45 testdata.511299 2026-02-19T13:27:14.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:14.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj46 testdata.511299 2026-02-19T13:27:15.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj47 testdata.511299 2026-02-19T13:27:15.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj48 testdata.511299 2026-02-19T13:27:15.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj49 testdata.511299 2026-02-19T13:27:15.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj50 testdata.511299 2026-02-19T13:27:15.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj51 testdata.511299 2026-02-19T13:27:15.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj52 testdata.511299 2026-02-19T13:27:15.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj53 testdata.511299 2026-02-19T13:27:15.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj54 testdata.511299 2026-02-19T13:27:15.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj55 testdata.511299 2026-02-19T13:27:15.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj56 testdata.511299 2026-02-19T13:27:15.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj57 testdata.511299 2026-02-19T13:27:15.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj58 testdata.511299 2026-02-19T13:27:15.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj59 testdata.511299 2026-02-19T13:27:15.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj60 testdata.511299 2026-02-19T13:27:15.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj61 testdata.511299 2026-02-19T13:27:15.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj62 testdata.511299 2026-02-19T13:27:15.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj63 testdata.511299 2026-02-19T13:27:15.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj64 testdata.511299 2026-02-19T13:27:15.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj65 testdata.511299 2026-02-19T13:27:15.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj66 testdata.511299 2026-02-19T13:27:15.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj67 testdata.511299 2026-02-19T13:27:15.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj68 testdata.511299 2026-02-19T13:27:15.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj69 testdata.511299 2026-02-19T13:27:15.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj70 testdata.511299 2026-02-19T13:27:15.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj71 testdata.511299 2026-02-19T13:27:15.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj72 testdata.511299 2026-02-19T13:27:15.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj73 testdata.511299 2026-02-19T13:27:15.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj74 testdata.511299 2026-02-19T13:27:15.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj75 testdata.511299 2026-02-19T13:27:15.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj76 testdata.511299 2026-02-19T13:27:15.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj77 testdata.511299 2026-02-19T13:27:15.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj78 testdata.511299 2026-02-19T13:27:15.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj79 testdata.511299 2026-02-19T13:27:15.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj80 testdata.511299 2026-02-19T13:27:15.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj81 testdata.511299 2026-02-19T13:27:15.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj82 testdata.511299 2026-02-19T13:27:15.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj83 testdata.511299 2026-02-19T13:27:15.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj84 testdata.511299 2026-02-19T13:27:15.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj85 testdata.511299 2026-02-19T13:27:15.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj86 testdata.511299 2026-02-19T13:27:15.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj87 testdata.511299 2026-02-19T13:27:15.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj88 testdata.511299 2026-02-19T13:27:15.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj89 testdata.511299 2026-02-19T13:27:15.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:565: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-19T13:27:15.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:567: TEST_dump_scrub_schedule: rados -p test put obj90 testdata.511299 2026-02-19T13:27:16.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:569: TEST_dump_scrub_schedule: rm -f testdata.511299 2026-02-19T13:27:16.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:571: TEST_dump_scrub_schedule: local pgid=1.0 2026-02-19T13:27:16.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:573: TEST_dump_scrub_schedule: date +%Y-%m-%dT%H:%M:%S.%N%:z 2026-02-19T13:27:16.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:573: TEST_dump_scrub_schedule: local now_is=2026-02-19T13:27:16.004501313+00:00 2026-02-19T13:27:16.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:581: TEST_dump_scrub_schedule: expct_starting=(['query_active']='false' ['query_is_future']='true' ['query_schedule']='scrub scheduled') 2026-02-19T13:27:16.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:581: TEST_dump_scrub_schedule: declare -A expct_starting 2026-02-19T13:27:16.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:582: TEST_dump_scrub_schedule: declare -A sched_data 2026-02-19T13:27:16.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:583: TEST_dump_scrub_schedule: extract_published_sch 1.0 2026-02-19T13:27:16.004501313+00:00 2019-10-12T20:32:43.645168+0000 sched_data 2026-02-19T13:27:16.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:17: extract_published_sch: local pgn=1.0 2026-02-19T13:27:16.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:18: extract_published_sch: local -n dict=sched_data 2026-02-19T13:27:16.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:19: extract_published_sch: local current_time=2026-02-19T13:27:16.004501313+00:00 2026-02-19T13:27:16.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:20: extract_published_sch: local extra_time=2019-10-12T20:32:43.645168+0000 2026-02-19T13:27:16.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:21: extract_published_sch: local extr_dbg=2 2026-02-19T13:27:16.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:24: extract_published_sch: local saved_echo_flag=x 2026-02-19T13:27:16.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:25: extract_published_sch: set +x 2026-02-19T13:27:16.151 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:27:16.160 INFO:tasks.workunit.client.0.trial144.stdout:from pg dump pg: { 2026-02-19T13:27:16.161 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_pg_state": "active+clean", 2026-02-19T13:27:16.161 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_state_has_scrubbing": false, 2026-02-19T13:27:16.161 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_last_duration": 0, 2026-02-19T13:27:16.161 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-19T13:27:16.161 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule_at": "2026-02-20T13:27:10.133418+0000", 2026-02-19T13:27:16.161 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_is_future": true, 2026-02-19T13:27:16.161 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_vs_date": true, 2026-02-19T13:27:16.161 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_reported_epoch": 19, 2026-02-19T13:27:16.161 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_seq": 18 2026-02-19T13:27:16.161 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:16.162 INFO:tasks.workunit.client.0.trial144.stdout:query output: 2026-02-19T13:27:16.234 INFO:tasks.workunit.client.0.trial144.stdout: "scrubber": { 2026-02-19T13:27:16.234 INFO:tasks.workunit.client.0.trial144.stdout: "active": false, 2026-02-19T13:27:16.234 INFO:tasks.workunit.client.0.trial144.stdout: "must_scrub": false, 2026-02-19T13:27:16.234 INFO:tasks.workunit.client.0.trial144.stdout: "must_deep_scrub": false, 2026-02-19T13:27:16.234 INFO:tasks.workunit.client.0.trial144.stdout: "must_repair": false, 2026-02-19T13:27:16.234 INFO:tasks.workunit.client.0.trial144.stdout: "scrub_reg_stamp": "2026-02-20T13:27:10.133418+0000", 2026-02-19T13:27:16.235 INFO:tasks.workunit.client.0.trial144.stdout: "schedule": "scrub scheduled @ 2026-02-20T13:27:10.133 (2026-02-20T13:27:10.133)" 2026-02-19T13:27:16.235 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:16.235 INFO:tasks.workunit.client.0.trial144.stdout: "agent_state": {} 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:( 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[query_epoch]=19 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[query_seq]=109 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[query_active]=false 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule]='scrub scheduled' 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule_at]='2026-02-20T13:27:10.133' 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[query_target_at]='2026-02-20T13:27:10.133' 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_duration]=0 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_stamp]='2026-02-19T13:27:10.133418+0000' 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_scrub]='0x0' 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[query_is_future]=true 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[query_vs_date]=true 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[query_scrub_seq]=null 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_pg_state]='active+clean' 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_state_has_scrubbing]=false 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_last_duration]=0 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-19T13:27:16.319 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule_at]='2026-02-20T13:27:10.133418+0000' 2026-02-19T13:27:16.320 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_is_future]=true 2026-02-19T13:27:16.320 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_vs_date]=true 2026-02-19T13:27:16.320 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_reported_epoch]=19 2026-02-19T13:27:16.320 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_seq]=18 2026-02-19T13:27:16.320 INFO:tasks.workunit.client.0.trial144.stdout:) 2026-02-19T13:27:16.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:584: TEST_dump_scrub_schedule: schedule_against_expected sched_data expct_starting initial 2026-02-19T13:27:16.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:178: schedule_against_expected: local -n dict=sched_data 2026-02-19T13:27:16.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:179: schedule_against_expected: local -n ep=expct_starting 2026-02-19T13:27:16.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:180: schedule_against_expected: local extr_dbg=1 2026-02-19T13:27:16.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:183: schedule_against_expected: local saved_echo_flag=x 2026-02-19T13:27:16.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:184: schedule_against_expected: set +x 2026-02-19T13:27:16.331 INFO:tasks.workunit.client.0.trial144.stdout:-- - comparing: 2026-02-19T13:27:16.331 INFO:tasks.workunit.client.0.trial144.stdout:key is query_schedule expected: scrub scheduled in actual: scrub scheduled 2026-02-19T13:27:16.331 INFO:tasks.workunit.client.0.trial144.stdout:key is query_is_future expected: true in actual: true 2026-02-19T13:27:16.331 INFO:tasks.workunit.client.0.trial144.stdout:key is query_active expected: false in actual: false 2026-02-19T13:27:16.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:203: schedule_against_expected: return 0 2026-02-19T13:27:16.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:585: TEST_dump_scrub_schedule: (( 0 == 0 )) 2026-02-19T13:27:16.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:586: TEST_dump_scrub_schedule: echo 'last-scrub --- ' 0x0 2026-02-19T13:27:16.331 INFO:tasks.workunit.client.0.trial144.stdout:last-scrub --- 0x0 2026-02-19T13:27:16.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:592: TEST_dump_scrub_schedule: saved_last_stamp=2026-02-19T13:27:10.133418+0000 2026-02-19T13:27:16.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:593: TEST_dump_scrub_schedule: ceph tell 'osd.*' config set osd_scrub_sleep 0 2026-02-19T13:27:16.395 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:27:16.396 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:27:16.396 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:16.403 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:27:16.403 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:27:16.403 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:16.410 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:27:16.410 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:27:16.410 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:16.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:594: TEST_dump_scrub_schedule: ceph tell 1.0 deep-scrub 2026-02-19T13:27:16.493 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:27:16.493 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T13:27:16.493 INFO:tasks.workunit.client.0.trial144.stdout: "must": true, 2026-02-19T13:27:16.493 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "0.000000" 2026-02-19T13:27:16.494 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:16.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:599: TEST_dump_scrub_schedule: sleep 5 2026-02-19T13:27:21.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:600: TEST_dump_scrub_schedule: sched_data=() 2026-02-19T13:27:21.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:601: TEST_dump_scrub_schedule: expct_qry_duration=(['query_last_duration']='0' ['query_last_duration_neg']='not0') 2026-02-19T13:27:21.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:601: TEST_dump_scrub_schedule: declare -A expct_qry_duration 2026-02-19T13:27:21.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:602: TEST_dump_scrub_schedule: wait_any_cond 1.0 10 2026-02-19T13:27:10.133418+0000 expct_qry_duration 'WaitingAfterScrub ' sched_data 2026-02-19T13:27:21.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.0 2026-02-19T13:27:21.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-19T13:27:21.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-19T13:27:10.133418+0000 2026-02-19T13:27:21.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=expct_qry_duration 2026-02-19T13:27:21.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sched_data 2026-02-19T13:27:21.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-19T13:27:21.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-19T13:27:21.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-19T13:27:21.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-19T13:27:21.504 INFO:tasks.workunit.client.0.trial144.stdout:waiting for any condition (WaitingAfterScrub ): pg:1.0 dt:2026-02-19T13:27:10.133418+0000 (10 retries) 2026-02-19T13:27:22.155 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:27:22.165 INFO:tasks.workunit.client.0.trial144.stdout:from pg dump pg: { 2026-02-19T13:27:22.165 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_pg_state": "active+clean", 2026-02-19T13:27:22.165 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_state_has_scrubbing": false, 2026-02-19T13:27:22.165 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_last_duration": 1, 2026-02-19T13:27:22.165 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-19T13:27:22.165 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule_at": "2026-02-20T13:27:16.990253+0000", 2026-02-19T13:27:22.165 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_is_future": true, 2026-02-19T13:27:22.165 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_vs_date": true, 2026-02-19T13:27:22.166 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_reported_epoch": 19, 2026-02-19T13:27:22.166 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_seq": 125 2026-02-19T13:27:22.166 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:22.166 INFO:tasks.workunit.client.0.trial144.stdout:query output: 2026-02-19T13:27:22.238 INFO:tasks.workunit.client.0.trial144.stdout: "scrubber": { 2026-02-19T13:27:22.238 INFO:tasks.workunit.client.0.trial144.stdout: "active": false, 2026-02-19T13:27:22.238 INFO:tasks.workunit.client.0.trial144.stdout: "must_scrub": false, 2026-02-19T13:27:22.238 INFO:tasks.workunit.client.0.trial144.stdout: "must_deep_scrub": false, 2026-02-19T13:27:22.238 INFO:tasks.workunit.client.0.trial144.stdout: "must_repair": false, 2026-02-19T13:27:22.238 INFO:tasks.workunit.client.0.trial144.stdout: "scrub_reg_stamp": "2026-02-20T13:27:16.990253+0000", 2026-02-19T13:27:22.239 INFO:tasks.workunit.client.0.trial144.stdout: "schedule": "scrub scheduled @ 2026-02-20T13:27:16.990 (2026-02-20T13:27:16.990)" 2026-02-19T13:27:22.239 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:22.239 INFO:tasks.workunit.client.0.trial144.stdout: "agent_state": {} 2026-02-19T13:27:22.322 INFO:tasks.workunit.client.0.trial144.stdout:( 2026-02-19T13:27:22.322 INFO:tasks.workunit.client.0.trial144.stdout:[query_epoch]=19 2026-02-19T13:27:22.322 INFO:tasks.workunit.client.0.trial144.stdout:[query_seq]=127 2026-02-19T13:27:22.322 INFO:tasks.workunit.client.0.trial144.stdout:[query_active]=false 2026-02-19T13:27:22.322 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule]='scrub scheduled' 2026-02-19T13:27:22.323 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule_at]='2026-02-20T13:27:16.990' 2026-02-19T13:27:22.323 INFO:tasks.workunit.client.0.trial144.stdout:[query_target_at]='2026-02-20T13:27:16.990' 2026-02-19T13:27:22.323 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_duration]=1 2026-02-19T13:27:22.323 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_stamp]='2026-02-19T13:27:16.990253+0000' 2026-02-19T13:27:22.323 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_scrub]='19x90' 2026-02-19T13:27:22.323 INFO:tasks.workunit.client.0.trial144.stdout:[query_is_future]=true 2026-02-19T13:27:22.323 INFO:tasks.workunit.client.0.trial144.stdout:[query_vs_date]=true 2026-02-19T13:27:22.323 INFO:tasks.workunit.client.0.trial144.stdout:[query_scrub_seq]=null 2026-02-19T13:27:22.323 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_pg_state]='active+clean' 2026-02-19T13:27:22.323 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_state_has_scrubbing]=false 2026-02-19T13:27:22.323 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_last_duration]=1 2026-02-19T13:27:22.323 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-19T13:27:22.324 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule_at]='2026-02-20T13:27:16.990253+0000' 2026-02-19T13:27:22.324 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_is_future]=true 2026-02-19T13:27:22.324 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_vs_date]=true 2026-02-19T13:27:22.324 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_reported_epoch]=19 2026-02-19T13:27:22.324 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_seq]=125 2026-02-19T13:27:22.324 INFO:tasks.workunit.client.0.trial144.stdout:) 2026-02-19T13:27:22.334 INFO:tasks.workunit.client.0.trial144.stdout:--> loop: 1 ~ false / 127 / 125 / true / 2026-02-19T13:27:16.990253+0000 / scrub scheduled %%% query_last_duration_neg query_last_duration 2026-02-19T13:27:22.334 INFO:tasks.workunit.client.0.trial144.stdout:key is query_last_duration: negation:1 # expected: 0 # in actual: 1 2026-02-19T13:27:22.334 INFO:tasks.workunit.client.0.trial144.stdout:WaitingAfterScrub - 'query_last_duration' actual value (1) matches expected (0) (negation: 1) 2026-02-19T13:27:22.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-19T13:27:22.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:604: TEST_dump_scrub_schedule: sched_data=() 2026-02-19T13:27:22.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:605: TEST_dump_scrub_schedule: expct_dmp_duration=(['dmp_last_duration']='0' ['dmp_last_duration_neg']='not0') 2026-02-19T13:27:22.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:605: TEST_dump_scrub_schedule: declare -A expct_dmp_duration 2026-02-19T13:27:22.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:606: TEST_dump_scrub_schedule: wait_any_cond 1.0 10 2026-02-19T13:27:10.133418+0000 expct_dmp_duration 'WaitingAfterScrub_dmp ' sched_data 2026-02-19T13:27:22.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.0 2026-02-19T13:27:22.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-19T13:27:22.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-19T13:27:10.133418+0000 2026-02-19T13:27:22.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=expct_dmp_duration 2026-02-19T13:27:22.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sched_data 2026-02-19T13:27:22.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-19T13:27:22.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-19T13:27:22.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-19T13:27:22.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-19T13:27:22.337 INFO:tasks.workunit.client.0.trial144.stdout:waiting for any condition (WaitingAfterScrub_dmp ): pg:1.0 dt:2026-02-19T13:27:10.133418+0000 (10 retries) 2026-02-19T13:27:22.984 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:27:22.994 INFO:tasks.workunit.client.0.trial144.stdout:from pg dump pg: { 2026-02-19T13:27:22.994 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_pg_state": "active+clean", 2026-02-19T13:27:22.994 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_state_has_scrubbing": false, 2026-02-19T13:27:22.994 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_last_duration": 1, 2026-02-19T13:27:22.994 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-19T13:27:22.994 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule_at": "2026-02-20T13:27:16.990253+0000", 2026-02-19T13:27:22.994 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_is_future": true, 2026-02-19T13:27:22.994 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_vs_date": true, 2026-02-19T13:27:22.995 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_reported_epoch": 19, 2026-02-19T13:27:22.995 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_seq": 126 2026-02-19T13:27:22.995 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:22.995 INFO:tasks.workunit.client.0.trial144.stdout:query output: 2026-02-19T13:27:23.066 INFO:tasks.workunit.client.0.trial144.stdout: "scrubber": { 2026-02-19T13:27:23.067 INFO:tasks.workunit.client.0.trial144.stdout: "active": false, 2026-02-19T13:27:23.067 INFO:tasks.workunit.client.0.trial144.stdout: "must_scrub": false, 2026-02-19T13:27:23.067 INFO:tasks.workunit.client.0.trial144.stdout: "must_deep_scrub": false, 2026-02-19T13:27:23.067 INFO:tasks.workunit.client.0.trial144.stdout: "must_repair": false, 2026-02-19T13:27:23.067 INFO:tasks.workunit.client.0.trial144.stdout: "scrub_reg_stamp": "2026-02-20T13:27:16.990253+0000", 2026-02-19T13:27:23.067 INFO:tasks.workunit.client.0.trial144.stdout: "schedule": "scrub scheduled @ 2026-02-20T13:27:16.990 (2026-02-20T13:27:16.990)" 2026-02-19T13:27:23.067 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:23.067 INFO:tasks.workunit.client.0.trial144.stdout: "agent_state": {} 2026-02-19T13:27:23.151 INFO:tasks.workunit.client.0.trial144.stdout:( 2026-02-19T13:27:23.151 INFO:tasks.workunit.client.0.trial144.stdout:[query_epoch]=19 2026-02-19T13:27:23.151 INFO:tasks.workunit.client.0.trial144.stdout:[query_seq]=127 2026-02-19T13:27:23.152 INFO:tasks.workunit.client.0.trial144.stdout:[query_active]=false 2026-02-19T13:27:23.152 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule]='scrub scheduled' 2026-02-19T13:27:23.152 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule_at]='2026-02-20T13:27:16.990' 2026-02-19T13:27:23.152 INFO:tasks.workunit.client.0.trial144.stdout:[query_target_at]='2026-02-20T13:27:16.990' 2026-02-19T13:27:23.152 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_duration]=1 2026-02-19T13:27:23.152 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_stamp]='2026-02-19T13:27:16.990253+0000' 2026-02-19T13:27:23.152 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_scrub]='19x90' 2026-02-19T13:27:23.152 INFO:tasks.workunit.client.0.trial144.stdout:[query_is_future]=true 2026-02-19T13:27:23.152 INFO:tasks.workunit.client.0.trial144.stdout:[query_vs_date]=true 2026-02-19T13:27:23.152 INFO:tasks.workunit.client.0.trial144.stdout:[query_scrub_seq]=null 2026-02-19T13:27:23.152 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_pg_state]='active+clean' 2026-02-19T13:27:23.152 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_state_has_scrubbing]=false 2026-02-19T13:27:23.153 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_last_duration]=1 2026-02-19T13:27:23.153 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-19T13:27:23.153 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule_at]='2026-02-20T13:27:16.990253+0000' 2026-02-19T13:27:23.153 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_is_future]=true 2026-02-19T13:27:23.153 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_vs_date]=true 2026-02-19T13:27:23.153 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_reported_epoch]=19 2026-02-19T13:27:23.153 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_seq]=126 2026-02-19T13:27:23.153 INFO:tasks.workunit.client.0.trial144.stdout:) 2026-02-19T13:27:23.163 INFO:tasks.workunit.client.0.trial144.stdout:--> loop: 1 ~ false / 127 / 126 / true / 2026-02-19T13:27:16.990253+0000 / scrub scheduled %%% dmp_last_duration dmp_last_duration_neg 2026-02-19T13:27:23.163 INFO:tasks.workunit.client.0.trial144.stdout:key is dmp_last_duration: negation:1 # expected: 0 # in actual: 1 2026-02-19T13:27:23.163 INFO:tasks.workunit.client.0.trial144.stdout:WaitingAfterScrub_dmp - 'dmp_last_duration' actual value (1) matches expected (0) (negation: 1) 2026-02-19T13:27:23.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-19T13:27:23.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:613: TEST_dump_scrub_schedule: ceph osd set noscrub 2026-02-19T13:27:23.367 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:27:23.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:614: TEST_dump_scrub_schedule: sleep 2 2026-02-19T13:27:25.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:615: TEST_dump_scrub_schedule: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_max 3 2026-02-19T13:27:25.442 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:27:25.442 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_max = '' " 2026-02-19T13:27:25.443 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:25.450 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:27:25.450 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_max = '' " 2026-02-19T13:27:25.450 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:25.457 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:27:25.457 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_shallow_scrub_chunk_max = '' " 2026-02-19T13:27:25.457 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:25.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:616: TEST_dump_scrub_schedule: ceph tell 'osd.*' config set osd_scrub_sleep 2.0 2026-02-19T13:27:25.540 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:27:25.540 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:27:25.540 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:25.547 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:27:25.547 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:27:25.548 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:25.555 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:27:25.555 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_sleep = '' " 2026-02-19T13:27:25.555 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:25.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:617: TEST_dump_scrub_schedule: sleep 8 2026-02-19T13:27:33.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:618: TEST_dump_scrub_schedule: saved_last_stamp=2026-02-19T13:27:16.990253+0000 2026-02-19T13:27:33.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:620: TEST_dump_scrub_schedule: ceph tell 1.0 schedule-scrub 2026-02-19T13:27:33.628 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:27:33.628 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:27:33.628 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:27:33.628 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-17T13:27:33.629284+0000" 2026-02-19T13:27:33.628 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:33.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:621: TEST_dump_scrub_schedule: sleep 1 2026-02-19T13:27:34.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:622: TEST_dump_scrub_schedule: sched_data=() 2026-02-19T13:27:34.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:623: TEST_dump_scrub_schedule: expct_scrub_peri_sched=(['query_is_future']='false') 2026-02-19T13:27:34.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:623: TEST_dump_scrub_schedule: declare -A expct_scrub_peri_sched 2026-02-19T13:27:34.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:624: TEST_dump_scrub_schedule: wait_any_cond 1.0 10 2026-02-19T13:27:16.990253+0000 expct_scrub_peri_sched waitingBeingScheduled sched_data 2026-02-19T13:27:34.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.0 2026-02-19T13:27:34.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-19T13:27:34.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-19T13:27:16.990253+0000 2026-02-19T13:27:34.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=expct_scrub_peri_sched 2026-02-19T13:27:34.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sched_data 2026-02-19T13:27:34.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-19T13:27:34.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-19T13:27:34.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-19T13:27:34.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-19T13:27:34.638 INFO:tasks.workunit.client.0.trial144.stdout:waiting for any condition (waitingBeingScheduled): pg:1.0 dt:2026-02-19T13:27:16.990253+0000 (10 retries) 2026-02-19T13:27:35.288 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:27:35.298 INFO:tasks.workunit.client.0.trial144.stdout:from pg dump pg: { 2026-02-19T13:27:35.298 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_pg_state": "active+clean", 2026-02-19T13:27:35.298 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_state_has_scrubbing": false, 2026-02-19T13:27:35.298 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_last_duration": 1, 2026-02-19T13:27:35.298 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule": "queued for scrub", 2026-02-19T13:27:35.298 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule_at": "0", 2026-02-19T13:27:35.298 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_is_future": false, 2026-02-19T13:27:35.298 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_vs_date": false, 2026-02-19T13:27:35.298 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_reported_epoch": 20, 2026-02-19T13:27:35.299 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_seq": 136 2026-02-19T13:27:35.299 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:35.299 INFO:tasks.workunit.client.0.trial144.stdout:query output: 2026-02-19T13:27:35.373 INFO:tasks.workunit.client.0.trial144.stdout: "scrubber": { 2026-02-19T13:27:35.373 INFO:tasks.workunit.client.0.trial144.stdout: "active": false, 2026-02-19T13:27:35.373 INFO:tasks.workunit.client.0.trial144.stdout: "must_scrub": false, 2026-02-19T13:27:35.373 INFO:tasks.workunit.client.0.trial144.stdout: "must_deep_scrub": false, 2026-02-19T13:27:35.373 INFO:tasks.workunit.client.0.trial144.stdout: "must_repair": false, 2026-02-19T13:27:35.373 INFO:tasks.workunit.client.0.trial144.stdout: "scrub_reg_stamp": "2026-02-19T13:27:35.050814+0000", 2026-02-19T13:27:35.373 INFO:tasks.workunit.client.0.trial144.stdout: "schedule": "queued for scrub at 2026-02-18T13:27:33.629" 2026-02-19T13:27:35.373 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:35.373 INFO:tasks.workunit.client.0.trial144.stdout: "agent_state": {} 2026-02-19T13:27:35.456 INFO:tasks.workunit.client.0.trial144.stdout:( 2026-02-19T13:27:35.456 INFO:tasks.workunit.client.0.trial144.stdout:[query_epoch]=20 2026-02-19T13:27:35.456 INFO:tasks.workunit.client.0.trial144.stdout:[query_seq]=137 2026-02-19T13:27:35.456 INFO:tasks.workunit.client.0.trial144.stdout:[query_active]=false 2026-02-19T13:27:35.456 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule]='queued for scrub at 2026-02-18T13:27:33.629' 2026-02-19T13:27:35.456 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule_at]='0' 2026-02-19T13:27:35.456 INFO:tasks.workunit.client.0.trial144.stdout:[query_target_at]='0' 2026-02-19T13:27:35.456 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_duration]=1 2026-02-19T13:27:35.456 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_stamp]='2026-02-17T13:27:33.629284+0000' 2026-02-19T13:27:35.457 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_scrub]='19x90' 2026-02-19T13:27:35.457 INFO:tasks.workunit.client.0.trial144.stdout:[query_is_future]=false 2026-02-19T13:27:35.457 INFO:tasks.workunit.client.0.trial144.stdout:[query_vs_date]=false 2026-02-19T13:27:35.457 INFO:tasks.workunit.client.0.trial144.stdout:[query_scrub_seq]=null 2026-02-19T13:27:35.457 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_pg_state]='active+clean' 2026-02-19T13:27:35.457 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_state_has_scrubbing]=false 2026-02-19T13:27:35.457 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_last_duration]=1 2026-02-19T13:27:35.457 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule]='queued for scrub' 2026-02-19T13:27:35.457 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule_at]='0' 2026-02-19T13:27:35.457 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_is_future]=false 2026-02-19T13:27:35.457 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_vs_date]=false 2026-02-19T13:27:35.457 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_reported_epoch]=20 2026-02-19T13:27:35.458 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_seq]=136 2026-02-19T13:27:35.458 INFO:tasks.workunit.client.0.trial144.stdout:) 2026-02-19T13:27:35.467 INFO:tasks.workunit.client.0.trial144.stdout:--> loop: 1 ~ false / 137 / 136 / false / 2026-02-17T13:27:33.629284+0000 / queued for scrub at 2026-02-18T13:27:33.629 %%% query_is_future 2026-02-19T13:27:35.467 INFO:tasks.workunit.client.0.trial144.stdout:key is query_is_future: negation:0 # expected: false # in actual: false 2026-02-19T13:27:35.467 INFO:tasks.workunit.client.0.trial144.stdout:waitingBeingScheduled - 'query_is_future' actual value (false) matches expected (false) (negation: 0) 2026-02-19T13:27:35.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-19T13:27:35.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:636: TEST_dump_scrub_schedule: saved_last_stamp=2026-02-17T13:27:33.629284+0000 2026-02-19T13:27:35.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:637: TEST_dump_scrub_schedule: ceph osd unset noscrub 2026-02-19T13:27:35.668 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is unset 2026-02-19T13:27:35.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:639: TEST_dump_scrub_schedule: cond_active=(['query_active']='true') 2026-02-19T13:27:35.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:639: TEST_dump_scrub_schedule: declare -A cond_active 2026-02-19T13:27:35.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:640: TEST_dump_scrub_schedule: sched_data=() 2026-02-19T13:27:35.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:641: TEST_dump_scrub_schedule: wait_any_cond 1.0 10 2026-02-17T13:27:33.629284+0000 cond_active 'WaitingActive ' sched_data 2026-02-19T13:27:35.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.0 2026-02-19T13:27:35.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-19T13:27:35.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-17T13:27:33.629284+0000 2026-02-19T13:27:35.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=cond_active 2026-02-19T13:27:35.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sched_data 2026-02-19T13:27:35.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-19T13:27:35.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-19T13:27:35.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-19T13:27:35.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-19T13:27:35.681 INFO:tasks.workunit.client.0.trial144.stdout:waiting for any condition (WaitingActive ): pg:1.0 dt:2026-02-17T13:27:33.629284+0000 (10 retries) 2026-02-19T13:27:36.331 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:27:36.343 INFO:tasks.workunit.client.0.trial144.stdout:from pg dump pg: { 2026-02-19T13:27:36.343 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_pg_state": "active+clean", 2026-02-19T13:27:36.343 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_state_has_scrubbing": false, 2026-02-19T13:27:36.343 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_last_duration": 1, 2026-02-19T13:27:36.344 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule": "queued for scrub", 2026-02-19T13:27:36.344 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule_at": "0", 2026-02-19T13:27:36.344 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_is_future": false, 2026-02-19T13:27:36.344 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_vs_date": false, 2026-02-19T13:27:36.344 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_reported_epoch": 20, 2026-02-19T13:27:36.344 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_seq": 136 2026-02-19T13:27:36.344 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:36.344 INFO:tasks.workunit.client.0.trial144.stdout:query output: 2026-02-19T13:27:36.415 INFO:tasks.workunit.client.0.trial144.stdout: "scrubber": { 2026-02-19T13:27:36.415 INFO:tasks.workunit.client.0.trial144.stdout: "active": true, 2026-02-19T13:27:36.415 INFO:tasks.workunit.client.0.trial144.stdout: "epoch_start": "16", 2026-02-19T13:27:36.415 INFO:tasks.workunit.client.0.trial144.stdout: "start": "1:00000000::::0", 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "end": "MIN", 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "max_end": "MIN", 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "subset_last_update": "0'0", 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "req_scrub": false, 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "auto_repair": false, 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "check_repair": false, 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "deep_scrub_on_error": false, 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "priority": 5, 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "shallow_errors": 0, 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "deep_errors": 0, 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "fixed": 0, 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "waiting_on_whom": [], 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "schedule": "scrubbing", 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "is_reserving_replicas": false 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:36.416 INFO:tasks.workunit.client.0.trial144.stdout: "agent_state": {} 2026-02-19T13:27:36.499 INFO:tasks.workunit.client.0.trial144.stdout:( 2026-02-19T13:27:36.499 INFO:tasks.workunit.client.0.trial144.stdout:[query_epoch]=21 2026-02-19T13:27:36.500 INFO:tasks.workunit.client.0.trial144.stdout:[query_seq]=143 2026-02-19T13:27:36.500 INFO:tasks.workunit.client.0.trial144.stdout:[query_active]=true 2026-02-19T13:27:36.500 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule]='scrubbing' 2026-02-19T13:27:36.500 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule_at]='0' 2026-02-19T13:27:36.500 INFO:tasks.workunit.client.0.trial144.stdout:[query_target_at]='0' 2026-02-19T13:27:36.500 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_duration]=1 2026-02-19T13:27:36.500 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_stamp]='2026-02-17T13:27:33.629284+0000' 2026-02-19T13:27:36.500 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_scrub]='19x90' 2026-02-19T13:27:36.500 INFO:tasks.workunit.client.0.trial144.stdout:[query_is_future]=false 2026-02-19T13:27:36.500 INFO:tasks.workunit.client.0.trial144.stdout:[query_vs_date]=false 2026-02-19T13:27:36.500 INFO:tasks.workunit.client.0.trial144.stdout:[query_scrub_seq]=null 2026-02-19T13:27:36.501 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_pg_state]='active+clean' 2026-02-19T13:27:36.501 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_state_has_scrubbing]=false 2026-02-19T13:27:36.501 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_last_duration]=1 2026-02-19T13:27:36.501 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule]='queued for scrub' 2026-02-19T13:27:36.501 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule_at]='0' 2026-02-19T13:27:36.501 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_is_future]=false 2026-02-19T13:27:36.501 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_vs_date]=false 2026-02-19T13:27:36.501 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_reported_epoch]=20 2026-02-19T13:27:36.501 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_seq]=136 2026-02-19T13:27:36.501 INFO:tasks.workunit.client.0.trial144.stdout:) 2026-02-19T13:27:36.511 INFO:tasks.workunit.client.0.trial144.stdout:--> loop: 1 ~ true / 143 / 136 / false / 2026-02-17T13:27:33.629284+0000 / scrubbing %%% query_active 2026-02-19T13:27:36.511 INFO:tasks.workunit.client.0.trial144.stdout:key is query_active: negation:0 # expected: true # in actual: true 2026-02-19T13:27:36.511 INFO:tasks.workunit.client.0.trial144.stdout:WaitingActive - 'query_active' actual value (true) matches expected (true) (negation: 0) 2026-02-19T13:27:36.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-19T13:27:36.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:645: TEST_dump_scrub_schedule: cond_active_dmp=(['dmp_state_has_scrubbing']='true' ['query_active']='false') 2026-02-19T13:27:36.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:645: TEST_dump_scrub_schedule: declare -A cond_active_dmp 2026-02-19T13:27:36.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:646: TEST_dump_scrub_schedule: sched_data=() 2026-02-19T13:27:36.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:647: TEST_dump_scrub_schedule: wait_any_cond 1.0 10 2026-02-17T13:27:33.629284+0000 cond_active_dmp 'WaitingActive ' sched_data 2026-02-19T13:27:36.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.0 2026-02-19T13:27:36.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-19T13:27:36.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-17T13:27:33.629284+0000 2026-02-19T13:27:36.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=cond_active_dmp 2026-02-19T13:27:36.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sched_data 2026-02-19T13:27:36.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-19T13:27:36.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-19T13:27:36.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-19T13:27:36.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-19T13:27:36.514 INFO:tasks.workunit.client.0.trial144.stdout:waiting for any condition (WaitingActive ): pg:1.0 dt:2026-02-17T13:27:33.629284+0000 (10 retries) 2026-02-19T13:27:37.167 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:27:37.179 INFO:tasks.workunit.client.0.trial144.stdout:from pg dump pg: { 2026-02-19T13:27:37.179 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_pg_state": "active+clean", 2026-02-19T13:27:37.179 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_state_has_scrubbing": false, 2026-02-19T13:27:37.179 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_last_duration": 1, 2026-02-19T13:27:37.180 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule": "queued for scrub", 2026-02-19T13:27:37.180 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule_at": "0", 2026-02-19T13:27:37.180 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_is_future": false, 2026-02-19T13:27:37.180 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_vs_date": false, 2026-02-19T13:27:37.180 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_reported_epoch": 21, 2026-02-19T13:27:37.180 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_seq": 139 2026-02-19T13:27:37.180 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:37.181 INFO:tasks.workunit.client.0.trial144.stdout:query output: 2026-02-19T13:27:37.252 INFO:tasks.workunit.client.0.trial144.stdout: "scrubber": { 2026-02-19T13:27:37.252 INFO:tasks.workunit.client.0.trial144.stdout: "active": true, 2026-02-19T13:27:37.252 INFO:tasks.workunit.client.0.trial144.stdout: "epoch_start": "16", 2026-02-19T13:27:37.252 INFO:tasks.workunit.client.0.trial144.stdout: "start": "1:00000000::::0", 2026-02-19T13:27:37.253 INFO:tasks.workunit.client.0.trial144.stdout: "end": "MIN", 2026-02-19T13:27:37.253 INFO:tasks.workunit.client.0.trial144.stdout: "max_end": "MIN", 2026-02-19T13:27:37.253 INFO:tasks.workunit.client.0.trial144.stdout: "subset_last_update": "0'0", 2026-02-19T13:27:37.253 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:27:37.253 INFO:tasks.workunit.client.0.trial144.stdout: "req_scrub": false, 2026-02-19T13:27:37.253 INFO:tasks.workunit.client.0.trial144.stdout: "auto_repair": false, 2026-02-19T13:27:37.253 INFO:tasks.workunit.client.0.trial144.stdout: "check_repair": false, 2026-02-19T13:27:37.253 INFO:tasks.workunit.client.0.trial144.stdout: "deep_scrub_on_error": false, 2026-02-19T13:27:37.253 INFO:tasks.workunit.client.0.trial144.stdout: "priority": 5, 2026-02-19T13:27:37.253 INFO:tasks.workunit.client.0.trial144.stdout: "shallow_errors": 0, 2026-02-19T13:27:37.253 INFO:tasks.workunit.client.0.trial144.stdout: "deep_errors": 0, 2026-02-19T13:27:37.253 INFO:tasks.workunit.client.0.trial144.stdout: "fixed": 0, 2026-02-19T13:27:37.254 INFO:tasks.workunit.client.0.trial144.stdout: "waiting_on_whom": [], 2026-02-19T13:27:37.254 INFO:tasks.workunit.client.0.trial144.stdout: "schedule": "scrubbing", 2026-02-19T13:27:37.254 INFO:tasks.workunit.client.0.trial144.stdout: "is_reserving_replicas": false 2026-02-19T13:27:37.254 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:37.254 INFO:tasks.workunit.client.0.trial144.stdout: "agent_state": {} 2026-02-19T13:27:37.338 INFO:tasks.workunit.client.0.trial144.stdout:( 2026-02-19T13:27:37.338 INFO:tasks.workunit.client.0.trial144.stdout:[query_epoch]=21 2026-02-19T13:27:37.338 INFO:tasks.workunit.client.0.trial144.stdout:[query_seq]=143 2026-02-19T13:27:37.338 INFO:tasks.workunit.client.0.trial144.stdout:[query_active]=true 2026-02-19T13:27:37.338 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule]='scrubbing' 2026-02-19T13:27:37.339 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule_at]='0' 2026-02-19T13:27:37.339 INFO:tasks.workunit.client.0.trial144.stdout:[query_target_at]='0' 2026-02-19T13:27:37.339 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_duration]=1 2026-02-19T13:27:37.339 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_stamp]='2026-02-17T13:27:33.629284+0000' 2026-02-19T13:27:37.339 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_scrub]='19x90' 2026-02-19T13:27:37.339 INFO:tasks.workunit.client.0.trial144.stdout:[query_is_future]=false 2026-02-19T13:27:37.339 INFO:tasks.workunit.client.0.trial144.stdout:[query_vs_date]=false 2026-02-19T13:27:37.339 INFO:tasks.workunit.client.0.trial144.stdout:[query_scrub_seq]=null 2026-02-19T13:27:37.339 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_pg_state]='active+clean' 2026-02-19T13:27:37.339 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_state_has_scrubbing]=false 2026-02-19T13:27:37.339 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_last_duration]=1 2026-02-19T13:27:37.340 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule]='queued for scrub' 2026-02-19T13:27:37.340 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule_at]='0' 2026-02-19T13:27:37.340 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_is_future]=false 2026-02-19T13:27:37.340 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_vs_date]=false 2026-02-19T13:27:37.340 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_reported_epoch]=21 2026-02-19T13:27:37.340 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_seq]=139 2026-02-19T13:27:37.340 INFO:tasks.workunit.client.0.trial144.stdout:) 2026-02-19T13:27:37.348 INFO:tasks.workunit.client.0.trial144.stdout:--> loop: 1 ~ true / 143 / 139 / false / 2026-02-17T13:27:33.629284+0000 / scrubbing %%% dmp_state_has_scrubbing query_active 2026-02-19T13:27:37.348 INFO:tasks.workunit.client.0.trial144.stdout:key is dmp_state_has_scrubbing: negation:0 # expected: true # in actual: false 2026-02-19T13:27:37.348 INFO:tasks.workunit.client.0.trial144.stdout:key is query_active: negation:0 # expected: false # in actual: true 2026-02-19T13:27:37.994 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:27:38.004 INFO:tasks.workunit.client.0.trial144.stdout:from pg dump pg: { 2026-02-19T13:27:38.004 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_pg_state": "active+clean", 2026-02-19T13:27:38.004 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_state_has_scrubbing": false, 2026-02-19T13:27:38.004 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_last_duration": 1, 2026-02-19T13:27:38.004 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule": "queued for scrub", 2026-02-19T13:27:38.004 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule_at": "0", 2026-02-19T13:27:38.004 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_is_future": false, 2026-02-19T13:27:38.004 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_vs_date": false, 2026-02-19T13:27:38.005 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_reported_epoch": 21, 2026-02-19T13:27:38.005 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_seq": 139 2026-02-19T13:27:38.005 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:38.005 INFO:tasks.workunit.client.0.trial144.stdout:query output: 2026-02-19T13:27:38.075 INFO:tasks.workunit.client.0.trial144.stdout: "scrubber": { 2026-02-19T13:27:38.076 INFO:tasks.workunit.client.0.trial144.stdout: "active": true, 2026-02-19T13:27:38.076 INFO:tasks.workunit.client.0.trial144.stdout: "epoch_start": "16", 2026-02-19T13:27:38.076 INFO:tasks.workunit.client.0.trial144.stdout: "start": "1:a0217245:::obj31:0", 2026-02-19T13:27:38.076 INFO:tasks.workunit.client.0.trial144.stdout: "end": "1:a0217245:::obj31:0", 2026-02-19T13:27:38.076 INFO:tasks.workunit.client.0.trial144.stdout: "max_end": "1:a0217245:::obj31:0", 2026-02-19T13:27:38.076 INFO:tasks.workunit.client.0.trial144.stdout: "subset_last_update": "19'90", 2026-02-19T13:27:38.076 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:27:38.076 INFO:tasks.workunit.client.0.trial144.stdout: "req_scrub": false, 2026-02-19T13:27:38.076 INFO:tasks.workunit.client.0.trial144.stdout: "auto_repair": false, 2026-02-19T13:27:38.077 INFO:tasks.workunit.client.0.trial144.stdout: "check_repair": false, 2026-02-19T13:27:38.077 INFO:tasks.workunit.client.0.trial144.stdout: "deep_scrub_on_error": false, 2026-02-19T13:27:38.077 INFO:tasks.workunit.client.0.trial144.stdout: "priority": 5, 2026-02-19T13:27:38.077 INFO:tasks.workunit.client.0.trial144.stdout: "shallow_errors": 0, 2026-02-19T13:27:38.077 INFO:tasks.workunit.client.0.trial144.stdout: "deep_errors": 0, 2026-02-19T13:27:38.077 INFO:tasks.workunit.client.0.trial144.stdout: "fixed": 0, 2026-02-19T13:27:38.077 INFO:tasks.workunit.client.0.trial144.stdout: "waiting_on_whom": [], 2026-02-19T13:27:38.077 INFO:tasks.workunit.client.0.trial144.stdout: "schedule": "scrubbing", 2026-02-19T13:27:38.077 INFO:tasks.workunit.client.0.trial144.stdout: "is_reserving_replicas": false 2026-02-19T13:27:38.077 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:38.077 INFO:tasks.workunit.client.0.trial144.stdout: "agent_state": {} 2026-02-19T13:27:38.163 INFO:tasks.workunit.client.0.trial144.stdout:( 2026-02-19T13:27:38.163 INFO:tasks.workunit.client.0.trial144.stdout:[query_epoch]=21 2026-02-19T13:27:38.163 INFO:tasks.workunit.client.0.trial144.stdout:[query_seq]=145 2026-02-19T13:27:38.164 INFO:tasks.workunit.client.0.trial144.stdout:[query_active]=true 2026-02-19T13:27:38.164 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule]='scrubbing' 2026-02-19T13:27:38.164 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule_at]='0' 2026-02-19T13:27:38.164 INFO:tasks.workunit.client.0.trial144.stdout:[query_target_at]='0' 2026-02-19T13:27:38.164 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_duration]=1 2026-02-19T13:27:38.164 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_stamp]='2026-02-17T13:27:33.629284+0000' 2026-02-19T13:27:38.164 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_scrub]='19x90' 2026-02-19T13:27:38.164 INFO:tasks.workunit.client.0.trial144.stdout:[query_is_future]=false 2026-02-19T13:27:38.164 INFO:tasks.workunit.client.0.trial144.stdout:[query_vs_date]=false 2026-02-19T13:27:38.164 INFO:tasks.workunit.client.0.trial144.stdout:[query_scrub_seq]=null 2026-02-19T13:27:38.164 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_pg_state]='active+clean' 2026-02-19T13:27:38.164 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_state_has_scrubbing]=false 2026-02-19T13:27:38.164 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_last_duration]=1 2026-02-19T13:27:38.165 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule]='queued for scrub' 2026-02-19T13:27:38.165 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule_at]='0' 2026-02-19T13:27:38.165 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_is_future]=false 2026-02-19T13:27:38.165 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_vs_date]=false 2026-02-19T13:27:38.165 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_reported_epoch]=21 2026-02-19T13:27:38.165 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_seq]=139 2026-02-19T13:27:38.165 INFO:tasks.workunit.client.0.trial144.stdout:) 2026-02-19T13:27:38.175 INFO:tasks.workunit.client.0.trial144.stdout:--> loop: 2 ~ true / 145 / 139 / false / 2026-02-17T13:27:33.629284+0000 / scrubbing %%% dmp_state_has_scrubbing query_active 2026-02-19T13:27:38.175 INFO:tasks.workunit.client.0.trial144.stdout:key is dmp_state_has_scrubbing: negation:0 # expected: true # in actual: false 2026-02-19T13:27:38.175 INFO:tasks.workunit.client.0.trial144.stdout:key is query_active: negation:0 # expected: false # in actual: true 2026-02-19T13:27:38.822 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:27:38.832 INFO:tasks.workunit.client.0.trial144.stdout:from pg dump pg: { 2026-02-19T13:27:38.832 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_pg_state": "active+clean+scrubbing", 2026-02-19T13:27:38.833 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_state_has_scrubbing": true, 2026-02-19T13:27:38.833 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_last_duration": 1, 2026-02-19T13:27:38.833 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule": "scrubbing for 2s", 2026-02-19T13:27:38.833 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule_at": "0", 2026-02-19T13:27:38.833 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_is_future": false, 2026-02-19T13:27:38.833 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_vs_date": false, 2026-02-19T13:27:38.833 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_reported_epoch": 21, 2026-02-19T13:27:38.833 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_seq": 143 2026-02-19T13:27:38.833 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:38.833 INFO:tasks.workunit.client.0.trial144.stdout:query output: 2026-02-19T13:27:38.903 INFO:tasks.workunit.client.0.trial144.stdout: "scrubber": { 2026-02-19T13:27:38.903 INFO:tasks.workunit.client.0.trial144.stdout: "active": true, 2026-02-19T13:27:38.903 INFO:tasks.workunit.client.0.trial144.stdout: "epoch_start": "16", 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "start": "1:a0217245:::obj31:0", 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "end": "1:a0217245:::obj31:0", 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "max_end": "1:a0217245:::obj31:0", 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "subset_last_update": "19'90", 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "req_scrub": false, 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "auto_repair": false, 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "check_repair": false, 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "deep_scrub_on_error": false, 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "priority": 5, 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "shallow_errors": 0, 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "deep_errors": 0, 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "fixed": 0, 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "waiting_on_whom": [], 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "schedule": "scrubbing", 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "is_reserving_replicas": false 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:38.904 INFO:tasks.workunit.client.0.trial144.stdout: "agent_state": {} 2026-02-19T13:27:38.990 INFO:tasks.workunit.client.0.trial144.stdout:( 2026-02-19T13:27:38.991 INFO:tasks.workunit.client.0.trial144.stdout:[query_epoch]=21 2026-02-19T13:27:38.991 INFO:tasks.workunit.client.0.trial144.stdout:[query_seq]=145 2026-02-19T13:27:38.991 INFO:tasks.workunit.client.0.trial144.stdout:[query_active]=true 2026-02-19T13:27:38.991 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule]='scrubbing' 2026-02-19T13:27:38.991 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule_at]='0' 2026-02-19T13:27:38.991 INFO:tasks.workunit.client.0.trial144.stdout:[query_target_at]='0' 2026-02-19T13:27:38.991 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_duration]=1 2026-02-19T13:27:38.991 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_stamp]='2026-02-17T13:27:33.629284+0000' 2026-02-19T13:27:38.991 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_scrub]='19x90' 2026-02-19T13:27:38.991 INFO:tasks.workunit.client.0.trial144.stdout:[query_is_future]=false 2026-02-19T13:27:38.991 INFO:tasks.workunit.client.0.trial144.stdout:[query_vs_date]=false 2026-02-19T13:27:38.992 INFO:tasks.workunit.client.0.trial144.stdout:[query_scrub_seq]=null 2026-02-19T13:27:38.992 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_pg_state]='active+clean+scrubbing' 2026-02-19T13:27:38.992 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_state_has_scrubbing]=true 2026-02-19T13:27:38.992 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_last_duration]=1 2026-02-19T13:27:38.992 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule]='scrubbing for 2s' 2026-02-19T13:27:38.992 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule_at]='0' 2026-02-19T13:27:38.992 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_is_future]=false 2026-02-19T13:27:38.992 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_vs_date]=false 2026-02-19T13:27:38.992 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_reported_epoch]=21 2026-02-19T13:27:38.992 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_seq]=143 2026-02-19T13:27:38.992 INFO:tasks.workunit.client.0.trial144.stdout:) 2026-02-19T13:27:39.002 INFO:tasks.workunit.client.0.trial144.stdout:--> loop: 3 ~ true / 145 / 143 / false / 2026-02-17T13:27:33.629284+0000 / scrubbing %%% dmp_state_has_scrubbing query_active 2026-02-19T13:27:39.002 INFO:tasks.workunit.client.0.trial144.stdout:key is dmp_state_has_scrubbing: negation:0 # expected: true # in actual: true 2026-02-19T13:27:39.003 INFO:tasks.workunit.client.0.trial144.stdout:WaitingActive - 'dmp_state_has_scrubbing' actual value (true) matches expected (true) (negation: 0) 2026-02-19T13:27:39.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-19T13:27:39.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:648: TEST_dump_scrub_schedule: sleep 4 2026-02-19T13:27:43.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:649: TEST_dump_scrub_schedule: perf_counters td/osd-scrub-test 3 2026-02-19T13:27:43.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-19T13:27:43.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-19T13:27:43.005 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-19T13:27:43.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-19T13:27:43.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:27:43.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-19T13:27:43.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:27:43.080 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:43.081 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.082 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.082 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:27:43.082 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:43.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:27:43.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-19T13:27:43.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:27:43.153 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:27:43.153 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:27:43.153 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:43.153 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:43.153 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:27:43.154 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:27:43.154 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:43.154 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:43.154 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:43.154 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:43.154 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:43.154 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:43.154 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:43.154 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.154 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.154 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:43.154 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:27:43.155 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:43.155 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:43.155 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:27:43.155 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:27:43.155 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:43.155 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:43.155 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:43.155 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 7, 2026-02-19T13:27:43.155 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:43.155 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:43.155 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:43.155 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.156 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.156 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:43.156 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:27:43.156 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:43.156 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:43.156 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:27:43.156 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:27:43.156 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:43.156 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:43.156 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:43.156 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:43.156 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:43.156 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:43.157 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:43.157 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.157 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.157 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:43.157 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:27:43.157 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:43.157 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:43.157 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:27:43.157 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:27:43.157 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:43.157 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:43.157 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:43.157 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 2, 2026-02-19T13:27:43.158 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:43.158 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:43.158 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:43.158 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.158 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.158 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:27:43.158 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:43.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:27:43.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-19T13:27:43.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:27:43.225 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:27:43.225 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:27:43.225 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:43.225 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:43.225 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:27:43.225 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:27:43.225 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:43.225 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:43.226 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:43.226 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:43.226 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:43.226 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:43.226 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:43.226 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.226 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.226 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:43.226 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:27:43.226 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:43.226 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:43.226 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:27:43.227 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:27:43.227 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:43.227 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:43.227 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:43.227 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:43.227 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:43.227 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:43.227 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:43.227 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.227 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.227 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:43.227 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:27:43.227 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:43.228 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:43.228 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:27:43.228 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:27:43.228 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:43.228 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:43.228 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:43.228 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:43.228 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:43.228 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:43.228 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:43.228 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.228 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.228 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:27:43.229 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:27:43.229 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:27:43.229 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:27:43.229 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:27:43.229 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:27:43.229 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:27:43.229 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:27:43.229 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:27:43.229 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:27:43.229 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:27:43.229 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:27:43.229 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:27:43.229 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.230 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:27:43.230 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:27:43.230 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:43.230 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Teardown Test TEST_dump_scrub_schedule ------------------ 2026-02-19T13:27:43.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_dump_scrub_schedule ------------------' 2026-02-19T13:27:43.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-19T13:27:43.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:27:43.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:27:43.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:27:43.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:27:43.232 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:27:43.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:27:43.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:27:43.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:27:43.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:27:43.335 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:27:43.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:27:43.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:27:43.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:27:43.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:27:43.338 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:27:43.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:27:43.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:27:43.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:27:43.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:27:43.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:27:43.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:27:43.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:27:43.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:27:43.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:43.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:43.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:27:43.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:27:43.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:27:43.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_dump_scrub_schedule ------------------' 2026-02-19T13:27:43.358 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Complete Test TEST_dump_scrub_schedule ------------------ 2026-02-19T13:27:43.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-19T13:27:43.359 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Prepare Test TEST_interval_changes ------------------- 2026-02-19T13:27:43.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_interval_changes -------------------' 2026-02-19T13:27:43.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-19T13:27:43.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-test 2026-02-19T13:27:43.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-test 2026-02-19T13:27:43.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:27:43.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:27:43.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:27:43.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:27:43.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:27:43.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:27:43.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:27:43.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:27:43.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:27:43.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:27:43.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:27:43.363 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:27:43.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:27:43.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:27:43.364 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:27:43.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:27:43.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:27:43.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:27:43.366 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:27:43.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:27:43.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:27:43.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:27:43.370 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:27:43.370 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:43.370 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:43.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:27:43.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:27:43.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:27:43.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-test 2026-02-19T13:27:43.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:27:43.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:43.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:43.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.511299 2026-02-19T13:27:43.375 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:27:43.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:27:43.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:27:43.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-19T13:27:43.376 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Run Test TEST_interval_changes ----------------------- 2026-02-19T13:27:43.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_interval_changes -----------------------' 2026-02-19T13:27:43.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_interval_changes td/osd-scrub-test 2026-02-19T13:27:43.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: TEST_interval_changes: local poolname=test 2026-02-19T13:27:43.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:143: TEST_interval_changes: local OSDS=2 2026-02-19T13:27:43.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: TEST_interval_changes: local objects=10 2026-02-19T13:27:43.377 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:146: TEST_interval_changes: expr 24 '*' 60 '*' 60 2026-02-19T13:27:43.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:146: TEST_interval_changes: local day=86400 2026-02-19T13:27:43.378 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:147: TEST_interval_changes: expr 86400 '*' 7 2026-02-19T13:27:43.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:147: TEST_interval_changes: local week=604800 2026-02-19T13:27:43.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:148: TEST_interval_changes: local min_interval=86400 2026-02-19T13:27:43.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:149: TEST_interval_changes: local max_interval=604800 2026-02-19T13:27:43.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:150: TEST_interval_changes: local WAIT_FOR_UPDATE=15 2026-02-19T13:27:43.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:152: TEST_interval_changes: TESTDATA=testdata.511299 2026-02-19T13:27:43.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:155: TEST_interval_changes: run_mon td/osd-scrub-test a --osd_pool_default_size=2 2026-02-19T13:27:43.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-test 2026-02-19T13:27:43.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:27:43.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:27:43.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:27:43.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-test/a 2026-02-19T13:27:43.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=2 2026-02-19T13:27:43.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:27:43.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:27:43.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:27:43.401 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:27:43.401 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:43.401 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:43.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:27:43.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-19T13:27:43.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:27:43.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:27:43.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:27:43.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:27:43.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:27:43.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:27:43.432 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:27:43.432 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:27:43.432 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:27:43.432 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:27:43.433 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:43.433 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:43.433 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-mon.a.asok 2026-02-19T13:27:43.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:27:43.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.511299/ceph-mon.a.asok config get fsid 2026-02-19T13:27:43.487 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:27:43.487 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:27:43.488 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:27:43.488 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:27:43.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:27:43.489 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:27:43.489 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:27:43.489 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:27:43.489 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:27:43.489 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:43.490 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:43.490 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-mon.a.asok 2026-02-19T13:27:43.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:27:43.490 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.511299/ceph-mon.a.asok config get mon_host 2026-02-19T13:27:43.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:156: TEST_interval_changes: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-19T13:27:43.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-test 2026-02-19T13:27:43.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:27:43.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:27:43.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:27:43.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-test/x 2026-02-19T13:27:43.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:27:43.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:27:43.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:27:43.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:27:43.638 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:27:43.638 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:43.638 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:43.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:27:43.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:27:43.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-19T13:27:43.658 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:157: TEST_interval_changes: expr 2 - 1 2026-02-19T13:27:43.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:157: TEST_interval_changes: seq 0 1 2026-02-19T13:27:43.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:157: TEST_interval_changes: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:27:43.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:159: TEST_interval_changes: run_osd td/osd-scrub-test 0 --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 2026-02-19T13:27:43.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:27:43.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:27:43.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:27:43.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:27:43.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-19T13:27:43.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:27:43.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:27:43.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:27:43.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:27:43.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-19T13:27:43.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-19T13:27:43.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:27:43.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:27:43.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:27:43.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:27:43.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:27:43.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:27:43.668 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:27:43.669 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:43.669 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:43.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:27:43.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:27:43.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:27:43.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:27:43.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:27:43.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:27:43.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:27:43.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:27:43.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:27:43.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:27:43.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:27:43.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:27:43.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0' 2026-02-19T13:27:43.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-19T13:27:43.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:27:43.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8f24c828-abe8-4e10-9d1a-f580a9e5198a 2026-02-19T13:27:43.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8f24c828-abe8-4e10-9d1a-f580a9e5198a' 2026-02-19T13:27:43.671 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 8f24c828-abe8-4e10-9d1a-f580a9e5198a 2026-02-19T13:27:43.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:27:43.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDPD5dpOOtOKBAAApu/zEr5FcTqHudtFNp3SA== 2026-02-19T13:27:43.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDPD5dpOOtOKBAAApu/zEr5FcTqHudtFNp3SA=="}' 2026-02-19T13:27:43.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8f24c828-abe8-4e10-9d1a-f580a9e5198a -i td/osd-scrub-test/0/new.json 2026-02-19T13:27:43.765 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:27:43.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-19T13:27:43.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 --mkfs --key AQDPD5dpOOtOKBAAApu/zEr5FcTqHudtFNp3SA== --osd-uuid 8f24c828-abe8-4e10-9d1a-f580a9e5198a 2026-02-19T13:27:43.790 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:43.785+0000 7f8dd358ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:43.791 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:43.789+0000 7f8dd358ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:43.792 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:43.789+0000 7f8dd358ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:43.792 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:43.789+0000 7f8dd358ca40 -1 bdev(0x55bbda826c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:27:43.793 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:43.789+0000 7f8dd358ca40 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-19T13:27:44.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-19T13:27:44.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:27:44.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:27:44.564 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:27:44.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:27:44.769 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:27:44.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:27:44.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 2026-02-19T13:27:44.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:27:44.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:27:44.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:27:44.785 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:44.781+0000 7f88b3618a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:44.787 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:44.781+0000 7f88b3618a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:44.788 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:44.785+0000 7f88b3618a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:44.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:27:44.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:27:44.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:27:44.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:27:44.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:27:44.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:27:44.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:27:44.927 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:27:44.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:27:44.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:27:45.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:27:45.165 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:45.161+0000 7f88b3618a40 -1 Falling back to public interface 2026-02-19T13:27:45.761 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:45.757+0000 7f88b3618a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:27:46.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:27:46.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:27:46.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:27:46.087 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:27:46.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:27:46.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:27:46.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:27:47.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:27:47.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:27:47.249 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:27:47.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:27:47.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:27:47.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:27:47.407 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3977185285,v1:127.0.0.1:6803/3977185285] [v2:127.0.0.1:6804/3977185285,v1:127.0.0.1:6805/3977185285] exists,up 8f24c828-abe8-4e10-9d1a-f580a9e5198a 2026-02-19T13:27:47.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:27:47.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:27:47.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:27:47.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:157: TEST_interval_changes: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:27:47.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:159: TEST_interval_changes: run_osd td/osd-scrub-test 1 --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 2026-02-19T13:27:47.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:27:47.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:27:47.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:27:47.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:27:47.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-19T13:27:47.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:27:47.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:27:47.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:27:47.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:27:47.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-19T13:27:47.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-19T13:27:47.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:27:47.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:27:47.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:27:47.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:27:47.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:27:47.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:27:47.413 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:27:47.413 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:47.413 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:47.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:27:47.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:27:47.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:27:47.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:27:47.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:27:47.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:27:47.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:27:47.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:27:47.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:27:47.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:27:47.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:27:47.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:27:47.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0' 2026-02-19T13:27:47.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-19T13:27:47.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:27:47.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1db1cb2b-6782-48b9-aa89-a5a06c971381 2026-02-19T13:27:47.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 1db1cb2b-6782-48b9-aa89-a5a06c971381' 2026-02-19T13:27:47.416 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 1db1cb2b-6782-48b9-aa89-a5a06c971381 2026-02-19T13:27:47.416 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:27:47.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDTD5dpDGJLGRAA5d2PKNo9Zetu3RJSSJCCGA== 2026-02-19T13:27:47.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDTD5dpDGJLGRAA5d2PKNo9Zetu3RJSSJCCGA=="}' 2026-02-19T13:27:47.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1db1cb2b-6782-48b9-aa89-a5a06c971381 -i td/osd-scrub-test/1/new.json 2026-02-19T13:27:47.587 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:27:47.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-19T13:27:47.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 --mkfs --key AQDTD5dpDGJLGRAA5d2PKNo9Zetu3RJSSJCCGA== --osd-uuid 1db1cb2b-6782-48b9-aa89-a5a06c971381 2026-02-19T13:27:47.614 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:47.609+0000 7f81f6824a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:47.615 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:47.613+0000 7f81f6824a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:47.616 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:47.613+0000 7f81f6824a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:47.617 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:47.613+0000 7f81f6824a40 -1 bdev(0x55f993fbcc00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:27:47.617 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:47.613+0000 7f81f6824a40 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-19T13:27:48.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-19T13:27:48.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:27:48.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:27:48.247 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:27:48.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:27:48.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:27:48.449 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:27:48.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 2026-02-19T13:27:48.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:27:48.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:27:48.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:27:48.466 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:48.461+0000 7f4d1ed50a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:48.467 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:48.465+0000 7f4d1ed50a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:48.468 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:48.465+0000 7f4d1ed50a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:27:48.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:27:48.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:27:48.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:27:48.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:27:48.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:27:48.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:27:48.607 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:27:48.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:27:48.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:27:48.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:27:48.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:27:48.897 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:48.893+0000 7f4d1ed50a40 -1 Falling back to public interface 2026-02-19T13:27:49.476 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:27:49.473+0000 7f4d1ed50a40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:27:49.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:27:49.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:27:49.769 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:27:49.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:27:49.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:27:49.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:27:49.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:27:50.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:27:50.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:27:50.936 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:27:50.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:27:50.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:27:50.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:27:51.093 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3029142738,v1:127.0.0.1:6811/3029142738] [v2:127.0.0.1:6812/3029142738,v1:127.0.0.1:6813/3029142738] exists,up 1db1cb2b-6782-48b9-aa89-a5a06c971381 2026-02-19T13:27:51.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:27:51.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:27:51.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:27:51.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:163: TEST_interval_changes: create_pool test 1 1 2026-02-19T13:27:51.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-19T13:27:51.299 INFO:tasks.workunit.client.0.trial144.stderr:pool 'test' created 2026-02-19T13:27:51.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:27:52.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:164: TEST_interval_changes: wait_for_clean 2026-02-19T13:27:52.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:27:52.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:27:52.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:27:52.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:27:52.315 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:27:52.315 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:27:52.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:27:52.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:27:52.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:27:52.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:27:52.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:27:52.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:27:52.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:27:52.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:27:52.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:27:52.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:27:52.574 INFO:tasks.workunit.client.0.trial144.stderr:1' 2026-02-19T13:27:52.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:27:52.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:27:52.574 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:27:52.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-19T13:27:52.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-19T13:27:52.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-19T13:27:52.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:27:52.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:27:52.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705668 2026-02-19T13:27:52.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705668 2026-02-19T13:27:52.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-38654705668' 2026-02-19T13:27:52.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:27:52.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-19T13:27:52.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:27:52.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:27:52.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-19T13:27:52.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:27:52.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-19T13:27:52.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-19T13:27:52.722 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836487 2026-02-19T13:27:52.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:27:52.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T13:27:52.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:27:53.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:27:53.880 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:27:54.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T13:27:54.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:27:55.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-19T13:27:55.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:27:55.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836489 -lt 21474836487 2026-02-19T13:27:55.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:27:55.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705668 2026-02-19T13:27:55.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:27:55.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:27:55.200 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705668 2026-02-19T13:27:55.200 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:27:55.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705668 2026-02-19T13:27:55.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-19T13:27:55.202 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705668 2026-02-19T13:27:55.202 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:27:55.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705669 -lt 38654705668 2026-02-19T13:27:55.366 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:27:55.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:27:55.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:27:55.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:27:55.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:27:55.557 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:27:55.557 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:27:55.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:27:55.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:27:55.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:27:55.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:27:55.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:27:55.713 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:27:55.713 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:27:55.713 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:27:55.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:27:55.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:27:55.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:27:55.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:165: TEST_interval_changes: ceph osd dump 2026-02-19T13:27:55.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:165: TEST_interval_changes: grep '^pool.*['\'']test['\'']' 2026-02-19T13:27:55.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:165: TEST_interval_changes: awk '{ print $2 }' 2026-02-19T13:27:56.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:165: TEST_interval_changes: local poolid=1 2026-02-19T13:27:56.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:167: TEST_interval_changes: dd if=/dev/urandom of=testdata.511299 bs=1032 count=1 2026-02-19T13:27:56.061 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:27:56.061 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:27:56.061 INFO:tasks.workunit.client.0.trial144.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 5.7236e-05 s, 18.0 MB/s 2026-02-19T13:27:56.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:168: TEST_interval_changes: seq 1 10 2026-02-19T13:27:56.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:168: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-19T13:27:56.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:170: TEST_interval_changes: rados -p test put obj1 testdata.511299 2026-02-19T13:27:56.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:168: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-19T13:27:56.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:170: TEST_interval_changes: rados -p test put obj2 testdata.511299 2026-02-19T13:27:56.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:168: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-19T13:27:56.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:170: TEST_interval_changes: rados -p test put obj3 testdata.511299 2026-02-19T13:27:56.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:168: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-19T13:27:56.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:170: TEST_interval_changes: rados -p test put obj4 testdata.511299 2026-02-19T13:27:56.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:168: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-19T13:27:56.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:170: TEST_interval_changes: rados -p test put obj5 testdata.511299 2026-02-19T13:27:56.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:168: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-19T13:27:56.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:170: TEST_interval_changes: rados -p test put obj6 testdata.511299 2026-02-19T13:27:56.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:168: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-19T13:27:56.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:170: TEST_interval_changes: rados -p test put obj7 testdata.511299 2026-02-19T13:27:56.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:168: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-19T13:27:56.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:170: TEST_interval_changes: rados -p test put obj8 testdata.511299 2026-02-19T13:27:56.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:168: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-19T13:27:56.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:170: TEST_interval_changes: rados -p test put obj9 testdata.511299 2026-02-19T13:27:56.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:168: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-19T13:27:56.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:170: TEST_interval_changes: rados -p test put obj10 testdata.511299 2026-02-19T13:27:56.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:172: TEST_interval_changes: rm -f testdata.511299 2026-02-19T13:27:56.287 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:174: TEST_interval_changes: get_primary test obj1 2026-02-19T13:27:56.287 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=test 2026-02-19T13:27:56.287 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T13:27:56.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map test obj1 2026-02-19T13:27:56.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:27:56.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:174: TEST_interval_changes: local primary=1 2026-02-19T13:27:56.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: check_dump_scrubs 1 '1 day' 2026-02-19T13:27:56.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:132: check_dump_scrubs: local primary=1 2026-02-19T13:27:56.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:133: check_dump_scrubs: local 'sched_time_check=1 day' 2026-02-19T13:27:56.444 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: get_asok_path osd.1 2026-02-19T13:27:56.444 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:27:56.444 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:27:56.444 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:27:56.444 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:56.445 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:56.445 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-osd.1.asok 2026-02-19T13:27:56.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: CEPH_ARGS= 2026-02-19T13:27:56.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: ceph --admin-daemon /tmp/ceph-asok.511299/ceph-osd.1.asok dump_scrubs 2026-02-19T13:27:56.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: DS='[ 2026-02-19T13:27:56.506 INFO:tasks.workunit.client.0.trial144.stderr: { 2026-02-19T13:27:56.506 INFO:tasks.workunit.client.0.trial144.stderr: "pgid": "1.0", 2026-02-19T13:27:56.506 INFO:tasks.workunit.client.0.trial144.stderr: "sched_time": "2026-02-20T13:27:51.299646+0000", 2026-02-19T13:27:56.506 INFO:tasks.workunit.client.0.trial144.stderr: "orig_sched_time": "2026-02-20T13:27:51.299646+0000", 2026-02-19T13:27:56.506 INFO:tasks.workunit.client.0.trial144.stderr: "forced": false, 2026-02-19T13:27:56.506 INFO:tasks.workunit.client.0.trial144.stderr: "level": "shallow", 2026-02-19T13:27:56.506 INFO:tasks.workunit.client.0.trial144.stderr: "urgency": "periodic-regular", 2026-02-19T13:27:56.506 INFO:tasks.workunit.client.0.trial144.stderr: "eligible": false, 2026-02-19T13:27:56.506 INFO:tasks.workunit.client.0.trial144.stderr: "last_issue": "ok" 2026-02-19T13:27:56.506 INFO:tasks.workunit.client.0.trial144.stderr: }, 2026-02-19T13:27:56.506 INFO:tasks.workunit.client.0.trial144.stderr: { 2026-02-19T13:27:56.507 INFO:tasks.workunit.client.0.trial144.stderr: "pgid": "1.0", 2026-02-19T13:27:56.507 INFO:tasks.workunit.client.0.trial144.stderr: "sched_time": "2026-02-25T22:47:38.724308+0000", 2026-02-19T13:27:56.507 INFO:tasks.workunit.client.0.trial144.stderr: "orig_sched_time": "2026-02-25T22:47:38.724308+0000", 2026-02-19T13:27:56.507 INFO:tasks.workunit.client.0.trial144.stderr: "forced": false, 2026-02-19T13:27:56.507 INFO:tasks.workunit.client.0.trial144.stderr: "level": "deep", 2026-02-19T13:27:56.507 INFO:tasks.workunit.client.0.trial144.stderr: "urgency": "periodic-regular", 2026-02-19T13:27:56.507 INFO:tasks.workunit.client.0.trial144.stderr: "eligible": false, 2026-02-19T13:27:56.507 INFO:tasks.workunit.client.0.trial144.stderr: "last_issue": "ok" 2026-02-19T13:27:56.507 INFO:tasks.workunit.client.0.trial144.stderr: } 2026-02-19T13:27:56.507 INFO:tasks.workunit.client.0.trial144.stderr:]' 2026-02-19T13:27:56.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: jq '.[0].sched_time' 2026-02-19T13:27:56.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-20T13:27:51.299646+0000",' '"orig_sched_time":' '"2026-02-20T13:27:51.299646+0000",' '"forced":' false, '"level":' '"shallow",' '"urgency":' '"periodic-regular",' '"eligible":' false, '"last_issue":' '"ok"' '},' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-25T22:47:38.724308+0000",' '"orig_sched_time":' '"2026-02-25T22:47:38.724308+0000",' '"forced":' false, '"level":' '"deep",' '"urgency":' '"periodic-regular",' '"eligible":' false, '"last_issue":' '"ok"' '}' ']' 2026-02-19T13:27:56.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: eval 'SCHED_TIME="2026-02-20T13:27:51.299646+0000"' 2026-02-19T13:27:56.515 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: SCHED_TIME=2026-02-20T13:27:51.299646+0000 2026-02-19T13:27:56.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: echo 2026-02-20T13:27:51.299646+0000 2026-02-19T13:27:56.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-19T13:27:56.518 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: date +%Y-%m-%d -d 'now + 1 day' 2026-02-19T13:27:56.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: test 2026-02-20 = 2026-02-20 2026-02-19T13:27:56.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:180: TEST_interval_changes: get_asok_path osd.1 2026-02-19T13:27:56.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:27:56.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:27:56.520 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:27:56.521 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:27:56.521 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:27:56.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-osd.1.asok 2026-02-19T13:27:56.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:180: TEST_interval_changes: expr 86400 '*' 2 2026-02-19T13:27:56.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:180: TEST_interval_changes: CEPH_ARGS= 2026-02-19T13:27:56.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:180: TEST_interval_changes: ceph --admin-daemon /tmp/ceph-asok.511299/ceph-osd.1.asok config set osd_scrub_min_interval 172800 2026-02-19T13:27:56.576 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:27:56.576 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_min_interval = '' " 2026-02-19T13:27:56.576 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:27:56.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:181: TEST_interval_changes: sleep 15 2026-02-19T13:28:11.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:182: TEST_interval_changes: check_dump_scrubs 1 '2 days' 2026-02-19T13:28:11.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:132: check_dump_scrubs: local primary=1 2026-02-19T13:28:11.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:133: check_dump_scrubs: local 'sched_time_check=2 days' 2026-02-19T13:28:11.585 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: get_asok_path osd.1 2026-02-19T13:28:11.585 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:28:11.585 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:28:11.586 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:28:11.586 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:28:11.586 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:28:11.586 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-osd.1.asok 2026-02-19T13:28:11.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: CEPH_ARGS= 2026-02-19T13:28:11.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: ceph --admin-daemon /tmp/ceph-asok.511299/ceph-osd.1.asok dump_scrubs 2026-02-19T13:28:11.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: DS='[ 2026-02-19T13:28:11.645 INFO:tasks.workunit.client.0.trial144.stderr: { 2026-02-19T13:28:11.645 INFO:tasks.workunit.client.0.trial144.stderr: "pgid": "1.0", 2026-02-19T13:28:11.645 INFO:tasks.workunit.client.0.trial144.stderr: "sched_time": "2026-02-21T13:27:51.299646+0000", 2026-02-19T13:28:11.645 INFO:tasks.workunit.client.0.trial144.stderr: "orig_sched_time": "2026-02-21T13:27:51.299646+0000", 2026-02-19T13:28:11.645 INFO:tasks.workunit.client.0.trial144.stderr: "forced": false, 2026-02-19T13:28:11.645 INFO:tasks.workunit.client.0.trial144.stderr: "level": "shallow", 2026-02-19T13:28:11.645 INFO:tasks.workunit.client.0.trial144.stderr: "urgency": "periodic-regular", 2026-02-19T13:28:11.645 INFO:tasks.workunit.client.0.trial144.stderr: "eligible": false, 2026-02-19T13:28:11.645 INFO:tasks.workunit.client.0.trial144.stderr: "last_issue": "ok" 2026-02-19T13:28:11.645 INFO:tasks.workunit.client.0.trial144.stderr: }, 2026-02-19T13:28:11.645 INFO:tasks.workunit.client.0.trial144.stderr: { 2026-02-19T13:28:11.646 INFO:tasks.workunit.client.0.trial144.stderr: "pgid": "1.0", 2026-02-19T13:28:11.646 INFO:tasks.workunit.client.0.trial144.stderr: "sched_time": "2026-02-28T02:11:02.941933+0000", 2026-02-19T13:28:11.646 INFO:tasks.workunit.client.0.trial144.stderr: "orig_sched_time": "2026-02-28T02:11:02.941933+0000", 2026-02-19T13:28:11.646 INFO:tasks.workunit.client.0.trial144.stderr: "forced": false, 2026-02-19T13:28:11.646 INFO:tasks.workunit.client.0.trial144.stderr: "level": "deep", 2026-02-19T13:28:11.646 INFO:tasks.workunit.client.0.trial144.stderr: "urgency": "periodic-regular", 2026-02-19T13:28:11.646 INFO:tasks.workunit.client.0.trial144.stderr: "eligible": false, 2026-02-19T13:28:11.646 INFO:tasks.workunit.client.0.trial144.stderr: "last_issue": "ok" 2026-02-19T13:28:11.646 INFO:tasks.workunit.client.0.trial144.stderr: } 2026-02-19T13:28:11.646 INFO:tasks.workunit.client.0.trial144.stderr:]' 2026-02-19T13:28:11.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: jq '.[0].sched_time' 2026-02-19T13:28:11.647 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-21T13:27:51.299646+0000",' '"orig_sched_time":' '"2026-02-21T13:27:51.299646+0000",' '"forced":' false, '"level":' '"shallow",' '"urgency":' '"periodic-regular",' '"eligible":' false, '"last_issue":' '"ok"' '},' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-28T02:11:02.941933+0000",' '"orig_sched_time":' '"2026-02-28T02:11:02.941933+0000",' '"forced":' false, '"level":' '"deep",' '"urgency":' '"periodic-regular",' '"eligible":' false, '"last_issue":' '"ok"' '}' ']' 2026-02-19T13:28:11.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: eval 'SCHED_TIME="2026-02-21T13:27:51.299646+0000"' 2026-02-19T13:28:11.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: SCHED_TIME=2026-02-21T13:27:51.299646+0000 2026-02-19T13:28:11.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: echo 2026-02-21T13:27:51.299646+0000 2026-02-19T13:28:11.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-19T13:28:11.658 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: date +%Y-%m-%d -d 'now + 2 days' 2026-02-19T13:28:11.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: test 2026-02-21 = 2026-02-21 2026-02-19T13:28:11.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:185: TEST_interval_changes: get_asok_path osd.1 2026-02-19T13:28:11.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:28:11.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:28:11.660 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:28:11.660 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:28:11.660 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:28:11.660 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-osd.1.asok 2026-02-19T13:28:11.661 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:185: TEST_interval_changes: expr 604800 '*' 2 2026-02-19T13:28:11.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:185: TEST_interval_changes: CEPH_ARGS= 2026-02-19T13:28:11.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:185: TEST_interval_changes: ceph --admin-daemon /tmp/ceph-asok.511299/ceph-osd.1.asok config set osd_scrub_max_interval 1209600 2026-02-19T13:28:11.714 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:28:11.714 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_max_interval = '' " 2026-02-19T13:28:11.714 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:28:11.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:186: TEST_interval_changes: sleep 15 2026-02-19T13:28:26.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:187: TEST_interval_changes: check_dump_scrubs 1 '2 days' 2026-02-19T13:28:26.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:132: check_dump_scrubs: local primary=1 2026-02-19T13:28:26.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:133: check_dump_scrubs: local 'sched_time_check=2 days' 2026-02-19T13:28:26.722 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: get_asok_path osd.1 2026-02-19T13:28:26.722 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:28:26.722 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:28:26.723 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:28:26.723 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:28:26.723 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:28:26.723 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-osd.1.asok 2026-02-19T13:28:26.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: CEPH_ARGS= 2026-02-19T13:28:26.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: ceph --admin-daemon /tmp/ceph-asok.511299/ceph-osd.1.asok dump_scrubs 2026-02-19T13:28:26.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: DS='[ 2026-02-19T13:28:26.783 INFO:tasks.workunit.client.0.trial144.stderr: { 2026-02-19T13:28:26.783 INFO:tasks.workunit.client.0.trial144.stderr: "pgid": "1.0", 2026-02-19T13:28:26.784 INFO:tasks.workunit.client.0.trial144.stderr: "sched_time": "2026-02-21T13:27:51.299646+0000", 2026-02-19T13:28:26.784 INFO:tasks.workunit.client.0.trial144.stderr: "orig_sched_time": "2026-02-21T13:27:51.299646+0000", 2026-02-19T13:28:26.784 INFO:tasks.workunit.client.0.trial144.stderr: "forced": false, 2026-02-19T13:28:26.784 INFO:tasks.workunit.client.0.trial144.stderr: "level": "shallow", 2026-02-19T13:28:26.784 INFO:tasks.workunit.client.0.trial144.stderr: "urgency": "periodic-regular", 2026-02-19T13:28:26.784 INFO:tasks.workunit.client.0.trial144.stderr: "eligible": false, 2026-02-19T13:28:26.784 INFO:tasks.workunit.client.0.trial144.stderr: "last_issue": "ok" 2026-02-19T13:28:26.784 INFO:tasks.workunit.client.0.trial144.stderr: }, 2026-02-19T13:28:26.784 INFO:tasks.workunit.client.0.trial144.stderr: { 2026-02-19T13:28:26.784 INFO:tasks.workunit.client.0.trial144.stderr: "pgid": "1.0", 2026-02-19T13:28:26.785 INFO:tasks.workunit.client.0.trial144.stderr: "sched_time": "2026-02-27T18:57:30.528258+0000", 2026-02-19T13:28:26.785 INFO:tasks.workunit.client.0.trial144.stderr: "orig_sched_time": "2026-02-27T18:57:30.528258+0000", 2026-02-19T13:28:26.785 INFO:tasks.workunit.client.0.trial144.stderr: "forced": false, 2026-02-19T13:28:26.785 INFO:tasks.workunit.client.0.trial144.stderr: "level": "deep", 2026-02-19T13:28:26.785 INFO:tasks.workunit.client.0.trial144.stderr: "urgency": "periodic-regular", 2026-02-19T13:28:26.785 INFO:tasks.workunit.client.0.trial144.stderr: "eligible": false, 2026-02-19T13:28:26.785 INFO:tasks.workunit.client.0.trial144.stderr: "last_issue": "ok" 2026-02-19T13:28:26.785 INFO:tasks.workunit.client.0.trial144.stderr: } 2026-02-19T13:28:26.785 INFO:tasks.workunit.client.0.trial144.stderr:]' 2026-02-19T13:28:26.787 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-21T13:27:51.299646+0000",' '"orig_sched_time":' '"2026-02-21T13:27:51.299646+0000",' '"forced":' false, '"level":' '"shallow",' '"urgency":' '"periodic-regular",' '"eligible":' false, '"last_issue":' '"ok"' '},' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-27T18:57:30.528258+0000",' '"orig_sched_time":' '"2026-02-27T18:57:30.528258+0000",' '"forced":' false, '"level":' '"deep",' '"urgency":' '"periodic-regular",' '"eligible":' false, '"last_issue":' '"ok"' '}' ']' 2026-02-19T13:28:26.787 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: jq '.[0].sched_time' 2026-02-19T13:28:26.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: eval 'SCHED_TIME="2026-02-21T13:27:51.299646+0000"' 2026-02-19T13:28:26.795 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: SCHED_TIME=2026-02-21T13:27:51.299646+0000 2026-02-19T13:28:26.796 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: echo 2026-02-21T13:27:51.299646+0000 2026-02-19T13:28:26.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-19T13:28:26.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: date +%Y-%m-%d -d 'now + 2 days' 2026-02-19T13:28:26.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: test 2026-02-21 = 2026-02-21 2026-02-19T13:28:26.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:190: TEST_interval_changes: expr 86400 '*' 3 2026-02-19T13:28:26.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:190: TEST_interval_changes: ceph osd pool set test scrub_min_interval 259200 2026-02-19T13:28:27.002 INFO:tasks.workunit.client.0.trial144.stderr:set pool 1 scrub_min_interval to 259200 2026-02-19T13:28:27.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:191: TEST_interval_changes: sleep 15 2026-02-19T13:28:42.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:192: TEST_interval_changes: check_dump_scrubs 1 '3 days' 2026-02-19T13:28:42.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:132: check_dump_scrubs: local primary=1 2026-02-19T13:28:42.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:133: check_dump_scrubs: local 'sched_time_check=3 days' 2026-02-19T13:28:42.018 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: get_asok_path osd.1 2026-02-19T13:28:42.018 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:28:42.018 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:28:42.019 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:28:42.019 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:28:42.019 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:28:42.019 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-osd.1.asok 2026-02-19T13:28:42.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: CEPH_ARGS= 2026-02-19T13:28:42.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: ceph --admin-daemon /tmp/ceph-asok.511299/ceph-osd.1.asok dump_scrubs 2026-02-19T13:28:42.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: DS='[ 2026-02-19T13:28:42.079 INFO:tasks.workunit.client.0.trial144.stderr: { 2026-02-19T13:28:42.079 INFO:tasks.workunit.client.0.trial144.stderr: "pgid": "1.0", 2026-02-19T13:28:42.079 INFO:tasks.workunit.client.0.trial144.stderr: "sched_time": "2026-02-22T13:27:51.299646+0000", 2026-02-19T13:28:42.079 INFO:tasks.workunit.client.0.trial144.stderr: "orig_sched_time": "2026-02-22T13:27:51.299646+0000", 2026-02-19T13:28:42.079 INFO:tasks.workunit.client.0.trial144.stderr: "forced": false, 2026-02-19T13:28:42.079 INFO:tasks.workunit.client.0.trial144.stderr: "level": "shallow", 2026-02-19T13:28:42.080 INFO:tasks.workunit.client.0.trial144.stderr: "urgency": "periodic-regular", 2026-02-19T13:28:42.080 INFO:tasks.workunit.client.0.trial144.stderr: "eligible": false, 2026-02-19T13:28:42.080 INFO:tasks.workunit.client.0.trial144.stderr: "last_issue": "ok" 2026-02-19T13:28:42.080 INFO:tasks.workunit.client.0.trial144.stderr: }, 2026-02-19T13:28:42.080 INFO:tasks.workunit.client.0.trial144.stderr: { 2026-02-19T13:28:42.080 INFO:tasks.workunit.client.0.trial144.stderr: "pgid": "1.0", 2026-02-19T13:28:42.080 INFO:tasks.workunit.client.0.trial144.stderr: "sched_time": "2026-02-26T15:08:19.217390+0000", 2026-02-19T13:28:42.080 INFO:tasks.workunit.client.0.trial144.stderr: "orig_sched_time": "2026-02-26T15:08:19.217390+0000", 2026-02-19T13:28:42.080 INFO:tasks.workunit.client.0.trial144.stderr: "forced": false, 2026-02-19T13:28:42.080 INFO:tasks.workunit.client.0.trial144.stderr: "level": "deep", 2026-02-19T13:28:42.081 INFO:tasks.workunit.client.0.trial144.stderr: "urgency": "periodic-regular", 2026-02-19T13:28:42.081 INFO:tasks.workunit.client.0.trial144.stderr: "eligible": false, 2026-02-19T13:28:42.081 INFO:tasks.workunit.client.0.trial144.stderr: "last_issue": "ok" 2026-02-19T13:28:42.081 INFO:tasks.workunit.client.0.trial144.stderr: } 2026-02-19T13:28:42.081 INFO:tasks.workunit.client.0.trial144.stderr:]' 2026-02-19T13:28:42.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: jq '.[0].sched_time' 2026-02-19T13:28:42.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-22T13:27:51.299646+0000",' '"orig_sched_time":' '"2026-02-22T13:27:51.299646+0000",' '"forced":' false, '"level":' '"shallow",' '"urgency":' '"periodic-regular",' '"eligible":' false, '"last_issue":' '"ok"' '},' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-26T15:08:19.217390+0000",' '"orig_sched_time":' '"2026-02-26T15:08:19.217390+0000",' '"forced":' false, '"level":' '"deep",' '"urgency":' '"periodic-regular",' '"eligible":' false, '"last_issue":' '"ok"' '}' ']' 2026-02-19T13:28:42.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: eval 'SCHED_TIME="2026-02-22T13:27:51.299646+0000"' 2026-02-19T13:28:42.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: SCHED_TIME=2026-02-22T13:27:51.299646+0000 2026-02-19T13:28:42.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: echo 2026-02-22T13:27:51.299646+0000 2026-02-19T13:28:42.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-19T13:28:42.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: date +%Y-%m-%d -d 'now + 3 days' 2026-02-19T13:28:42.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: test 2026-02-22 = 2026-02-22 2026-02-19T13:28:42.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:195: TEST_interval_changes: expr 604800 '*' 3 2026-02-19T13:28:42.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:195: TEST_interval_changes: ceph osd pool set test scrub_max_interval 1814400 2026-02-19T13:28:42.281 INFO:tasks.workunit.client.0.trial144.stderr:set pool 1 scrub_max_interval to 1814400 2026-02-19T13:28:42.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:196: TEST_interval_changes: sleep 15 2026-02-19T13:28:57.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:197: TEST_interval_changes: check_dump_scrubs 1 '3 days' 2026-02-19T13:28:57.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:132: check_dump_scrubs: local primary=1 2026-02-19T13:28:57.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:133: check_dump_scrubs: local 'sched_time_check=3 days' 2026-02-19T13:28:57.296 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: get_asok_path osd.1 2026-02-19T13:28:57.296 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:28:57.296 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:28:57.297 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:28:57.297 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:28:57.297 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:28:57.298 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-osd.1.asok 2026-02-19T13:28:57.298 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: CEPH_ARGS= 2026-02-19T13:28:57.298 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: ceph --admin-daemon /tmp/ceph-asok.511299/ceph-osd.1.asok dump_scrubs 2026-02-19T13:28:57.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: DS='[ 2026-02-19T13:28:57.356 INFO:tasks.workunit.client.0.trial144.stderr: { 2026-02-19T13:28:57.356 INFO:tasks.workunit.client.0.trial144.stderr: "pgid": "1.0", 2026-02-19T13:28:57.356 INFO:tasks.workunit.client.0.trial144.stderr: "sched_time": "2026-02-22T13:27:51.299646+0000", 2026-02-19T13:28:57.356 INFO:tasks.workunit.client.0.trial144.stderr: "orig_sched_time": "2026-02-22T13:27:51.299646+0000", 2026-02-19T13:28:57.357 INFO:tasks.workunit.client.0.trial144.stderr: "forced": false, 2026-02-19T13:28:57.357 INFO:tasks.workunit.client.0.trial144.stderr: "level": "shallow", 2026-02-19T13:28:57.357 INFO:tasks.workunit.client.0.trial144.stderr: "urgency": "periodic-regular", 2026-02-19T13:28:57.357 INFO:tasks.workunit.client.0.trial144.stderr: "eligible": false, 2026-02-19T13:28:57.357 INFO:tasks.workunit.client.0.trial144.stderr: "last_issue": "ok" 2026-02-19T13:28:57.357 INFO:tasks.workunit.client.0.trial144.stderr: }, 2026-02-19T13:28:57.357 INFO:tasks.workunit.client.0.trial144.stderr: { 2026-02-19T13:28:57.357 INFO:tasks.workunit.client.0.trial144.stderr: "pgid": "1.0", 2026-02-19T13:28:57.357 INFO:tasks.workunit.client.0.trial144.stderr: "sched_time": "2026-02-25T21:21:06.515712+0000", 2026-02-19T13:28:57.357 INFO:tasks.workunit.client.0.trial144.stderr: "orig_sched_time": "2026-02-25T21:21:06.515712+0000", 2026-02-19T13:28:57.357 INFO:tasks.workunit.client.0.trial144.stderr: "forced": false, 2026-02-19T13:28:57.357 INFO:tasks.workunit.client.0.trial144.stderr: "level": "deep", 2026-02-19T13:28:57.358 INFO:tasks.workunit.client.0.trial144.stderr: "urgency": "periodic-regular", 2026-02-19T13:28:57.358 INFO:tasks.workunit.client.0.trial144.stderr: "eligible": false, 2026-02-19T13:28:57.358 INFO:tasks.workunit.client.0.trial144.stderr: "last_issue": "ok" 2026-02-19T13:28:57.358 INFO:tasks.workunit.client.0.trial144.stderr: } 2026-02-19T13:28:57.358 INFO:tasks.workunit.client.0.trial144.stderr:]' 2026-02-19T13:28:57.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-22T13:27:51.299646+0000",' '"orig_sched_time":' '"2026-02-22T13:27:51.299646+0000",' '"forced":' false, '"level":' '"shallow",' '"urgency":' '"periodic-regular",' '"eligible":' false, '"last_issue":' '"ok"' '},' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-25T21:21:06.515712+0000",' '"orig_sched_time":' '"2026-02-25T21:21:06.515712+0000",' '"forced":' false, '"level":' '"deep",' '"urgency":' '"periodic-regular",' '"eligible":' false, '"last_issue":' '"ok"' '}' ']' 2026-02-19T13:28:57.359 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: jq '.[0].sched_time' 2026-02-19T13:28:57.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: eval 'SCHED_TIME="2026-02-22T13:27:51.299646+0000"' 2026-02-19T13:28:57.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: SCHED_TIME=2026-02-22T13:27:51.299646+0000 2026-02-19T13:28:57.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: echo 2026-02-22T13:27:51.299646+0000 2026-02-19T13:28:57.368 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-19T13:28:57.370 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: date +%Y-%m-%d -d 'now + 3 days' 2026-02-19T13:28:57.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:138: check_dump_scrubs: test 2026-02-22 = 2026-02-22 2026-02-19T13:28:57.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:198: TEST_interval_changes: perf_counters td/osd-scrub-test 2 2026-02-19T13:28:57.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-19T13:28:57.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=2 2026-02-19T13:28:57.372 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 2 - 1 2026-02-19T13:28:57.373 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 1 2026-02-19T13:28:57.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:28:57.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-19T13:28:57.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:28:57.450 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:28:57.451 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:28:57.451 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:28:57.451 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:28:57.451 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:28:57.451 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:28:57.451 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:28:57.451 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:28:57.451 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:28:57.451 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:28:57.451 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:28:57.452 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:28:57.452 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:28:57.452 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.452 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.452 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:28:57.452 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:28:57.452 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:28:57.452 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:28:57.452 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:28:57.452 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:28:57.453 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:28:57.453 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:28:57.453 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:28:57.453 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:28:57.453 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:28:57.453 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:28:57.453 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:28:57.453 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.453 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.453 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:28:57.453 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:28:57.453 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:28:57.454 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:28:57.454 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:28:57.454 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:28:57.454 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:28:57.454 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:28:57.454 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:28:57.454 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:28:57.454 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:28:57.454 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:28:57.454 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:28:57.454 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.454 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.454 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:28:57.455 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:28:57.455 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:28:57.455 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:28:57.455 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:28:57.455 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:28:57.455 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:28:57.455 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:28:57.455 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:28:57.455 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:28:57.455 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:28:57.455 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:28:57.455 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:28:57.456 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.456 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.456 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:28:57.456 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:28:57.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:28:57.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-19T13:28:57.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:28:57.521 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:28:57.522 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:28:57.522 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:28:57.522 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:28:57.522 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:28:57.522 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:28:57.522 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:28:57.522 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:28:57.522 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:28:57.522 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:28:57.522 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:28:57.522 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:28:57.523 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:28:57.523 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.523 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.523 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:28:57.523 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:28:57.523 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:28:57.523 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:28:57.523 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:28:57.523 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:28:57.523 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:28:57.523 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:28:57.523 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:28:57.524 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:28:57.524 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:28:57.524 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:28:57.524 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:28:57.524 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.524 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.524 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:28:57.524 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:28:57.524 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:28:57.524 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:28:57.524 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:28:57.524 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:28:57.524 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:28:57.525 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:28:57.525 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:28:57.525 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:28:57.525 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:28:57.525 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:28:57.525 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:28:57.525 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.525 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.525 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:28:57.525 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:28:57.525 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:28:57.525 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:28:57.525 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:28:57.526 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:28:57.526 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:28:57.526 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:28:57.526 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:28:57.526 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:28:57.526 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:28:57.526 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:28:57.526 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:28:57.526 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.526 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:28:57.526 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:28:57.526 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:28:57.527 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Teardown Test TEST_interval_changes ------------------ 2026-02-19T13:28:57.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_interval_changes ------------------' 2026-02-19T13:28:57.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-19T13:28:57.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:28:57.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:28:57.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:28:57.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:28:57.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:28:57.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:28:57.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:28:57.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:28:57.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:28:57.632 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:28:57.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:28:57.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:28:57.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:28:57.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:28:57.635 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:28:57.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:28:57.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:28:57.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:28:57.637 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:28:57.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:28:57.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:28:57.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:28:57.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:28:57.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:28:57.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:28:57.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:28:57.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:28:57.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:28:57.648 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Complete Test TEST_interval_changes ------------------ 2026-02-19T13:28:57.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_interval_changes ------------------' 2026-02-19T13:28:57.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-19T13:28:57.649 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Prepare Test TEST_just_deep_scrubs ------------------- 2026-02-19T13:28:57.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_just_deep_scrubs -------------------' 2026-02-19T13:28:57.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-19T13:28:57.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-test 2026-02-19T13:28:57.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-test 2026-02-19T13:28:57.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:28:57.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:28:57.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:28:57.651 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:28:57.651 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:28:57.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:28:57.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:28:57.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:28:57.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:28:57.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:28:57.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:28:57.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:28:57.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:28:57.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:28:57.654 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:28:57.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:28:57.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:28:57.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:28:57.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:28:57.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:28:57.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:28:57.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:28:57.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:28:57.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:28:57.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:28:57.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:28:57.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:28:57.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:28:57.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-test 2026-02-19T13:28:57.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:28:57.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:28:57.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:28:57.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.511299 2026-02-19T13:28:57.664 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:28:57.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:28:57.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:28:57.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-19T13:28:57.665 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Run Test TEST_just_deep_scrubs ----------------------- 2026-02-19T13:28:57.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_just_deep_scrubs -----------------------' 2026-02-19T13:28:57.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_just_deep_scrubs td/osd-scrub-test 2026-02-19T13:28:57.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:462: TEST_just_deep_scrubs: local dir=td/osd-scrub-test 2026-02-19T13:28:57.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:463: TEST_just_deep_scrubs: cluster_conf=(['osds_num']='3' ['pgs_in_pool']='4' ['pool_name']='test') 2026-02-19T13:28:57.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:463: TEST_just_deep_scrubs: local -A cluster_conf 2026-02-19T13:28:57.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:469: TEST_just_deep_scrubs: standard_scrub_cluster td/osd-scrub-test cluster_conf 2026-02-19T13:28:57.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:229: standard_scrub_cluster: local dir=td/osd-scrub-test 2026-02-19T13:28:57.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:230: standard_scrub_cluster: local -n args=cluster_conf 2026-02-19T13:28:57.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:232: standard_scrub_cluster: local OSDS=3 2026-02-19T13:28:57.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:233: standard_scrub_cluster: local pg_num=4 2026-02-19T13:28:57.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:235: standard_scrub_cluster: local poolname=test 2026-02-19T13:28:57.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:236: standard_scrub_cluster: args['pool_name']=test 2026-02-19T13:28:57.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:238: standard_scrub_cluster: local pool_default_size=3 2026-02-19T13:28:57.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:239: standard_scrub_cluster: args['pool_default_size']=3 2026-02-19T13:28:57.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:241: standard_scrub_cluster: local extra_pars= 2026-02-19T13:28:57.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:242: standard_scrub_cluster: local debug_msg=dbg 2026-02-19T13:28:57.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:245: standard_scrub_cluster: local saved_echo_flag=x 2026-02-19T13:28:57.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:246: standard_scrub_cluster: set +x 2026-02-19T13:28:57.944 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 6bd748b4-7fac-47da-ae0b-6ab3de19c03c 2026-02-19T13:28:58.045 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:28:58.073 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:28:58.071+0000 7f25ceb1fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:28:58.074 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:28:58.071+0000 7f25ceb1fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:28:58.076 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:28:58.075+0000 7f25ceb1fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:28:58.076 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:28:58.075+0000 7f25ceb1fa40 -1 bdev(0x56419cad2c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:28:58.076 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:28:58.075+0000 7f25ceb1fa40 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-19T13:28:58.870 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:28:59.068 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:28:59.087 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:28:59.087+0000 7fb4f74cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:28:59.089 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:28:59.087+0000 7fb4f74cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:28:59.090 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:28:59.087+0000 7fb4f74cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:28:59.229 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:28:59.496 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:28:59.495+0000 7fb4f74cca40 -1 Falling back to public interface 2026-02-19T13:29:00.184 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:00.183+0000 7fb4f74cca40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:29:00.390 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:29:01.549 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:29:01.705 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1090566181,v1:127.0.0.1:6803/1090566181] [v2:127.0.0.1:6804/1090566181,v1:127.0.0.1:6805/1090566181] exists,up 6bd748b4-7fac-47da-ae0b-6ab3de19c03c 2026-02-19T13:29:01.710 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 084b5ceb-9c20-4267-86f8-cdc6c4a6c343 2026-02-19T13:29:01.884 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:29:01.915 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:01.911+0000 7f2cc4c2fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:01.916 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:01.915+0000 7f2cc4c2fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:01.917 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:01.915+0000 7f2cc4c2fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:01.918 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:01.915+0000 7f2cc4c2fa40 -1 bdev(0x55853c60ac00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:29:01.918 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:01.915+0000 7f2cc4c2fa40 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-19T13:29:02.607 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:29:02.814 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:29:02.845 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:02.843+0000 7f09169d2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:02.846 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:02.843+0000 7f09169d2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:02.848 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:02.847+0000 7f09169d2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:02.975 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:29:03.257 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:03.255+0000 7f09169d2a40 -1 Falling back to public interface 2026-02-19T13:29:03.845 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:03.843+0000 7f09169d2a40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:29:04.133 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:29:05.290 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:29:05.450 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3131941499,v1:127.0.0.1:6811/3131941499] [v2:127.0.0.1:6812/3131941499,v1:127.0.0.1:6813/3131941499] exists,up 084b5ceb-9c20-4267-86f8-cdc6c4a6c343 2026-02-19T13:29:05.455 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 2eea88da-e699-41cd-bf7f-be4b3303a45a 2026-02-19T13:29:05.627 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:29:05.664 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:05.663+0000 7f3ed1a7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:05.666 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:05.663+0000 7f3ed1a7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:05.667 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:05.667+0000 7f3ed1a7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:05.667 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:05.667+0000 7f3ed1a7fa40 -1 bdev(0x55a17dd3ac00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:29:05.667 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:05.667+0000 7f3ed1a7fa40 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-19T13:29:06.319 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T13:29:06.516 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T13:29:06.546 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:06.543+0000 7f0ae7345a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:06.548 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:06.547+0000 7f0ae7345a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:06.549 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:06.547+0000 7f0ae7345a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:06.678 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:29:06.940 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:06.939+0000 7f0ae7345a40 -1 Falling back to public interface 2026-02-19T13:29:07.507 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:07.503+0000 7f0ae7345a40 -1 osd.2 0 log_to_monitors true 2026-02-19T13:29:07.835 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:29:08.993 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:29:09.154 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/249465820,v1:127.0.0.1:6819/249465820] [v2:127.0.0.1:6820/249465820,v1:127.0.0.1:6821/249465820] exists,up 2eea88da-e699-41cd-bf7f-be4b3303a45a 2026-02-19T13:29:09.366 INFO:tasks.workunit.client.0.trial144.stderr:pool 'test' created 2026-02-19T13:29:10.864 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836491 2026-02-19T13:29:13.359 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705671 2026-02-19T13:29:13.534 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542149 2026-02-19T13:29:15.411 INFO:tasks.workunit.client.0.trial144.stdout:standard_scrub_cluster: dbg: test pool is test 1 2026-02-19T13:29:15.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:470: TEST_just_deep_scrubs: local poolid=1 2026-02-19T13:29:15.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:471: TEST_just_deep_scrubs: local poolname=test 2026-02-19T13:29:15.412 INFO:tasks.workunit.client.0.trial144.stdout:Pool: test : 1 2026-02-19T13:29:15.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:472: TEST_just_deep_scrubs: echo 'Pool: test : 1' 2026-02-19T13:29:15.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:474: TEST_just_deep_scrubs: TESTDATA=testdata.511299 2026-02-19T13:29:15.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:475: TEST_just_deep_scrubs: local objects=90 2026-02-19T13:29:15.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:476: TEST_just_deep_scrubs: dd if=/dev/urandom of=testdata.511299 bs=1032 count=1 2026-02-19T13:29:15.414 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:29:15.414 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:29:15.414 INFO:tasks.workunit.client.0.trial144.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.5356e-05 s, 29.2 MB/s 2026-02-19T13:29:15.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: seq 1 90 2026-02-19T13:29:15.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj1 testdata.511299 2026-02-19T13:29:15.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj2 testdata.511299 2026-02-19T13:29:15.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj3 testdata.511299 2026-02-19T13:29:15.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj4 testdata.511299 2026-02-19T13:29:15.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj5 testdata.511299 2026-02-19T13:29:15.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj6 testdata.511299 2026-02-19T13:29:15.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj7 testdata.511299 2026-02-19T13:29:15.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj8 testdata.511299 2026-02-19T13:29:15.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj9 testdata.511299 2026-02-19T13:29:15.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj10 testdata.511299 2026-02-19T13:29:15.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj11 testdata.511299 2026-02-19T13:29:15.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj12 testdata.511299 2026-02-19T13:29:15.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj13 testdata.511299 2026-02-19T13:29:15.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj14 testdata.511299 2026-02-19T13:29:15.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj15 testdata.511299 2026-02-19T13:29:15.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj16 testdata.511299 2026-02-19T13:29:15.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj17 testdata.511299 2026-02-19T13:29:15.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj18 testdata.511299 2026-02-19T13:29:15.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj19 testdata.511299 2026-02-19T13:29:15.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj20 testdata.511299 2026-02-19T13:29:15.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj21 testdata.511299 2026-02-19T13:29:15.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj22 testdata.511299 2026-02-19T13:29:15.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj23 testdata.511299 2026-02-19T13:29:15.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj24 testdata.511299 2026-02-19T13:29:15.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:15.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj25 testdata.511299 2026-02-19T13:29:16.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj26 testdata.511299 2026-02-19T13:29:16.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj27 testdata.511299 2026-02-19T13:29:16.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj28 testdata.511299 2026-02-19T13:29:16.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj29 testdata.511299 2026-02-19T13:29:16.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj30 testdata.511299 2026-02-19T13:29:16.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj31 testdata.511299 2026-02-19T13:29:16.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj32 testdata.511299 2026-02-19T13:29:16.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj33 testdata.511299 2026-02-19T13:29:16.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj34 testdata.511299 2026-02-19T13:29:16.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj35 testdata.511299 2026-02-19T13:29:16.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj36 testdata.511299 2026-02-19T13:29:16.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj37 testdata.511299 2026-02-19T13:29:16.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj38 testdata.511299 2026-02-19T13:29:16.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj39 testdata.511299 2026-02-19T13:29:16.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj40 testdata.511299 2026-02-19T13:29:16.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj41 testdata.511299 2026-02-19T13:29:16.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj42 testdata.511299 2026-02-19T13:29:16.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj43 testdata.511299 2026-02-19T13:29:16.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj44 testdata.511299 2026-02-19T13:29:16.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj45 testdata.511299 2026-02-19T13:29:16.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj46 testdata.511299 2026-02-19T13:29:16.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj47 testdata.511299 2026-02-19T13:29:16.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj48 testdata.511299 2026-02-19T13:29:16.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.577 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj49 testdata.511299 2026-02-19T13:29:16.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj50 testdata.511299 2026-02-19T13:29:16.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj51 testdata.511299 2026-02-19T13:29:16.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj52 testdata.511299 2026-02-19T13:29:16.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj53 testdata.511299 2026-02-19T13:29:16.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj54 testdata.511299 2026-02-19T13:29:16.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj55 testdata.511299 2026-02-19T13:29:16.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj56 testdata.511299 2026-02-19T13:29:16.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj57 testdata.511299 2026-02-19T13:29:16.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj58 testdata.511299 2026-02-19T13:29:16.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj59 testdata.511299 2026-02-19T13:29:16.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj60 testdata.511299 2026-02-19T13:29:16.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj61 testdata.511299 2026-02-19T13:29:16.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj62 testdata.511299 2026-02-19T13:29:16.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj63 testdata.511299 2026-02-19T13:29:16.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj64 testdata.511299 2026-02-19T13:29:16.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj65 testdata.511299 2026-02-19T13:29:16.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj66 testdata.511299 2026-02-19T13:29:16.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:16.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj67 testdata.511299 2026-02-19T13:29:17.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj68 testdata.511299 2026-02-19T13:29:17.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj69 testdata.511299 2026-02-19T13:29:17.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj70 testdata.511299 2026-02-19T13:29:17.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj71 testdata.511299 2026-02-19T13:29:17.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj72 testdata.511299 2026-02-19T13:29:17.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj73 testdata.511299 2026-02-19T13:29:17.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj74 testdata.511299 2026-02-19T13:29:17.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj75 testdata.511299 2026-02-19T13:29:17.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj76 testdata.511299 2026-02-19T13:29:17.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj77 testdata.511299 2026-02-19T13:29:17.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj78 testdata.511299 2026-02-19T13:29:17.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj79 testdata.511299 2026-02-19T13:29:17.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj80 testdata.511299 2026-02-19T13:29:17.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj81 testdata.511299 2026-02-19T13:29:17.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj82 testdata.511299 2026-02-19T13:29:17.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj83 testdata.511299 2026-02-19T13:29:17.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj84 testdata.511299 2026-02-19T13:29:17.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj85 testdata.511299 2026-02-19T13:29:17.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj86 testdata.511299 2026-02-19T13:29:17.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj87 testdata.511299 2026-02-19T13:29:17.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj88 testdata.511299 2026-02-19T13:29:17.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj89 testdata.511299 2026-02-19T13:29:17.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-19T13:29:17.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: rados -p test put obj90 testdata.511299 2026-02-19T13:29:17.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:481: TEST_just_deep_scrubs: rm -f testdata.511299 2026-02-19T13:29:17.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:486: TEST_just_deep_scrubs: ceph tell 'osd.*' config set osd_scrub_retry_after_noscrub 2 2026-02-19T13:29:17.642 INFO:tasks.workunit.client.0.trial144.stdout:osd.0: { 2026-02-19T13:29:17.642 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_retry_after_noscrub = '' (not observed, change may require restart) " 2026-02-19T13:29:17.642 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:29:17.649 INFO:tasks.workunit.client.0.trial144.stdout:osd.1: { 2026-02-19T13:29:17.649 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_retry_after_noscrub = '' (not observed, change may require restart) " 2026-02-19T13:29:17.649 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:29:17.656 INFO:tasks.workunit.client.0.trial144.stdout:osd.2: { 2026-02-19T13:29:17.656 INFO:tasks.workunit.client.0.trial144.stdout: "success": "osd_scrub_retry_after_noscrub = '' (not observed, change may require restart) " 2026-02-19T13:29:17.657 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:29:17.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:487: TEST_just_deep_scrubs: ceph osd set noscrub 2026-02-19T13:29:17.865 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:29:17.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:488: TEST_just_deep_scrubs: ceph osd set nodeep-scrub 2026-02-19T13:29:18.071 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is set 2026-02-19T13:29:18.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:489: TEST_just_deep_scrubs: sleep 6 2026-02-19T13:29:24.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:490: TEST_just_deep_scrubs: date -Ins 2026-02-19T13:29:24.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:490: TEST_just_deep_scrubs: local now_is=2026-02-19T13:29:24,086336990+00:00 2026-02-19T13:29:24.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:491: TEST_just_deep_scrubs: declare -A sched_data 2026-02-19T13:29:24.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:492: TEST_just_deep_scrubs: local pgid=1.2 2026-02-19T13:29:24.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:495: TEST_just_deep_scrubs: set_query_debug 1.2 2026-02-19T13:29:24.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:417: set_query_debug: local pgid=1.2 2026-02-19T13:29:24.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:419: set_query_debug: ceph pg dump pgs_brief 2026-02-19T13:29:24.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:419: set_query_debug: awk -v 'pg=^1.2' -n -e '$0 ~ pg { print(gensub(/[^0-9]*([0-9]+).*/,"\\1","g",$5)); }' 2026-02-19T13:29:24.231 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs_brief 2026-02-19T13:29:24.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:419: set_query_debug: local prim_osd=0 2026-02-19T13:29:24.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:421: set_query_debug: echo 'Setting scrub debug data. Primary for 1.2 is 0' 2026-02-19T13:29:24.241 INFO:tasks.workunit.client.0.trial144.stdout:Setting scrub debug data. Primary for 1.2 is 0 2026-02-19T13:29:24.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:422: set_query_debug: get_asok_path osd.0 2026-02-19T13:29:24.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:29:24.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:29:24.242 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:29:24.242 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:29:24.242 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:29:24.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-osd.0.asok 2026-02-19T13:29:24.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:422: set_query_debug: CEPH_ARGS= 2026-02-19T13:29:24.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:422: set_query_debug: ceph --format=json daemon /tmp/ceph-asok.511299/ceph-osd.0.asok scrubdebug 1.2 set sessions 2026-02-19T13:29:24.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:497: TEST_just_deep_scrubs: extract_published_sch 1.2 2026-02-19T13:29:24,086336990+00:00 2026-02-19T13:29:24,086336990+00:00 sched_data 2026-02-19T13:29:24.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:17: extract_published_sch: local pgn=1.2 2026-02-19T13:29:24.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:18: extract_published_sch: local -n dict=sched_data 2026-02-19T13:29:24.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:19: extract_published_sch: local current_time=2026-02-19T13:29:24,086336990+00:00 2026-02-19T13:29:24.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:20: extract_published_sch: local extra_time=2026-02-19T13:29:24,086336990+00:00 2026-02-19T13:29:24.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:21: extract_published_sch: local extr_dbg=2 2026-02-19T13:29:24.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:24: extract_published_sch: local saved_echo_flag=x 2026-02-19T13:29:24.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:25: extract_published_sch: set +x 2026-02-19T13:29:24.461 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:29:24.471 INFO:tasks.workunit.client.0.trial144.stdout:{"success":true}from pg dump pg: { 2026-02-19T13:29:24.471 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_pg_state": "active+clean", 2026-02-19T13:29:24.471 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_state_has_scrubbing": false, 2026-02-19T13:29:24.471 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_last_duration": 0, 2026-02-19T13:29:24.471 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-19T13:29:24.471 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule_at": "2026-02-20T13:29:09.366465+0000", 2026-02-19T13:29:24.471 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_is_future": true, 2026-02-19T13:29:24.471 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_vs_date": true, 2026-02-19T13:29:24.471 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_reported_epoch": 20, 2026-02-19T13:29:24.471 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_seq": 40 2026-02-19T13:29:24.471 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:29:24.471 INFO:tasks.workunit.client.0.trial144.stdout:query output: 2026-02-19T13:29:24.543 INFO:tasks.workunit.client.0.trial144.stdout: "scrubber": { 2026-02-19T13:29:24.543 INFO:tasks.workunit.client.0.trial144.stdout: "active": false, 2026-02-19T13:29:24.543 INFO:tasks.workunit.client.0.trial144.stdout: "must_scrub": false, 2026-02-19T13:29:24.543 INFO:tasks.workunit.client.0.trial144.stdout: "must_deep_scrub": false, 2026-02-19T13:29:24.543 INFO:tasks.workunit.client.0.trial144.stdout: "must_repair": false, 2026-02-19T13:29:24.543 INFO:tasks.workunit.client.0.trial144.stdout: "scrub_reg_stamp": "2026-02-20T13:29:09.366465+0000", 2026-02-19T13:29:24.544 INFO:tasks.workunit.client.0.trial144.stdout: "schedule": "scrub scheduled @ 2026-02-20T13:29:09.366 (2026-02-20T13:29:09.366)", 2026-02-19T13:29:24.544 INFO:tasks.workunit.client.0.trial144.stdout: "test_sequence": 7561216 2026-02-19T13:29:24.544 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:24.544 INFO:tasks.workunit.client.0.trial144.stdout: "agent_state": {} 2026-02-19T13:29:24.630 INFO:tasks.workunit.client.0.trial144.stdout:( 2026-02-19T13:29:24.631 INFO:tasks.workunit.client.0.trial144.stdout:[query_epoch]=20 2026-02-19T13:29:24.631 INFO:tasks.workunit.client.0.trial144.stdout:[query_seq]=41 2026-02-19T13:29:24.631 INFO:tasks.workunit.client.0.trial144.stdout:[query_active]=false 2026-02-19T13:29:24.631 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule]='scrub scheduled' 2026-02-19T13:29:24.631 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule_at]='2026-02-20T13:29:09.366' 2026-02-19T13:29:24.631 INFO:tasks.workunit.client.0.trial144.stdout:[query_target_at]='2026-02-20T13:29:09.366' 2026-02-19T13:29:24.631 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_duration]=0 2026-02-19T13:29:24.631 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_stamp]='2026-02-19T13:29:09.366465+0000' 2026-02-19T13:29:24.631 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_scrub]='0x0' 2026-02-19T13:29:24.631 INFO:tasks.workunit.client.0.trial144.stdout:[query_is_future]=true 2026-02-19T13:29:24.631 INFO:tasks.workunit.client.0.trial144.stdout:[query_vs_date]=true 2026-02-19T13:29:24.631 INFO:tasks.workunit.client.0.trial144.stdout:[query_scrub_seq]=7561216 2026-02-19T13:29:24.632 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_pg_state]='active+clean' 2026-02-19T13:29:24.632 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_state_has_scrubbing]=false 2026-02-19T13:29:24.632 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_last_duration]=0 2026-02-19T13:29:24.632 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-19T13:29:24.632 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule_at]='2026-02-20T13:29:09.366465+0000' 2026-02-19T13:29:24.632 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_is_future]=true 2026-02-19T13:29:24.632 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_vs_date]=true 2026-02-19T13:29:24.632 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_reported_epoch]=20 2026-02-19T13:29:24.632 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_seq]=40 2026-02-19T13:29:24.632 INFO:tasks.workunit.client.0.trial144.stdout:) 2026-02-19T13:29:24.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:498: TEST_just_deep_scrubs: local saved_last_stamp=2026-02-19T13:29:09.366465+0000 2026-02-19T13:29:24.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:499: TEST_just_deep_scrubs: local dbg_counter_at_start=7561216 2026-02-19T13:29:24.643 INFO:tasks.workunit.client.0.trial144.stdout:test counter @ start: 7561216 2026-02-19T13:29:24.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:500: TEST_just_deep_scrubs: echo 'test counter @ start: 7561216' 2026-02-19T13:29:24.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:502: TEST_just_deep_scrubs: ceph tell 1.2 schedule-deep-scrub 2026-02-19T13:29:24.706 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:29:24.706 INFO:tasks.workunit.client.0.trial144.stdout: "deep": true, 2026-02-19T13:29:24.706 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:29:24.707 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-08T08:41:14.708006+0000" 2026-02-19T13:29:24.707 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:29:24.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:504: TEST_just_deep_scrubs: sleep 5 2026-02-19T13:29:29.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:506: TEST_just_deep_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-19T13:29:29.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:506: TEST_just_deep_scrubs: jq -r '.pg_stats[] | "\(.pgid) \(.stat_sum.num_objects)"' 2026-02-19T13:29:29.864 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:29:29.872 INFO:tasks.workunit.client.0.trial144.stdout:1.3 20 2026-02-19T13:29:29.872 INFO:tasks.workunit.client.0.trial144.stdout:1.2 20 2026-02-19T13:29:29.873 INFO:tasks.workunit.client.0.trial144.stdout:1.1 25 2026-02-19T13:29:29.873 INFO:tasks.workunit.client.0.trial144.stdout:1.0 25 2026-02-19T13:29:29.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:507: TEST_just_deep_scrubs: ceph pg 1.2 query --format=json-pretty 2026-02-19T13:29:29.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:507: TEST_just_deep_scrubs: jq -r .info.stats.stat_sum.num_objects 2026-02-19T13:29:29.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:507: TEST_just_deep_scrubs: echo 'Objects # in pg 1.2: ' 20 2026-02-19T13:29:29.948 INFO:tasks.workunit.client.0.trial144.stdout:Objects # in pg 1.2: 20 2026-02-19T13:29:29.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:509: TEST_just_deep_scrubs: declare -A sc_data_2 2026-02-19T13:29:29.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:510: TEST_just_deep_scrubs: extract_published_sch 1.2 2026-02-19T13:29:24,086336990+00:00 2026-02-19T13:29:24,086336990+00:00 sc_data_2 2026-02-19T13:29:29.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:17: extract_published_sch: local pgn=1.2 2026-02-19T13:29:29.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:18: extract_published_sch: local -n dict=sc_data_2 2026-02-19T13:29:29.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:19: extract_published_sch: local current_time=2026-02-19T13:29:24,086336990+00:00 2026-02-19T13:29:29.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:20: extract_published_sch: local extra_time=2026-02-19T13:29:24,086336990+00:00 2026-02-19T13:29:29.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:21: extract_published_sch: local extr_dbg=2 2026-02-19T13:29:29.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:24: extract_published_sch: local saved_echo_flag=x 2026-02-19T13:29:29.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:25: extract_published_sch: set +x 2026-02-19T13:29:30.092 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:29:30.103 INFO:tasks.workunit.client.0.trial144.stdout:from pg dump pg: { 2026-02-19T13:29:30.103 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_pg_state": "active+clean", 2026-02-19T13:29:30.103 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_state_has_scrubbing": false, 2026-02-19T13:29:30.103 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_last_duration": 0, 2026-02-19T13:29:30.103 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule": "queued for deep scrub", 2026-02-19T13:29:30.103 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule_at": "0", 2026-02-19T13:29:30.103 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_is_future": false, 2026-02-19T13:29:30.103 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_vs_date": false, 2026-02-19T13:29:30.103 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_reported_epoch": 20, 2026-02-19T13:29:30.103 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_seq": 45 2026-02-19T13:29:30.103 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:29:30.103 INFO:tasks.workunit.client.0.trial144.stdout:query output: 2026-02-19T13:29:30.175 INFO:tasks.workunit.client.0.trial144.stdout: "scrubber": { 2026-02-19T13:29:30.175 INFO:tasks.workunit.client.0.trial144.stdout: "active": false, 2026-02-19T13:29:30.175 INFO:tasks.workunit.client.0.trial144.stdout: "must_scrub": false, 2026-02-19T13:29:30.175 INFO:tasks.workunit.client.0.trial144.stdout: "must_deep_scrub": false, 2026-02-19T13:29:30.175 INFO:tasks.workunit.client.0.trial144.stdout: "must_repair": false, 2026-02-19T13:29:30.175 INFO:tasks.workunit.client.0.trial144.stdout: "scrub_reg_stamp": "2026-02-19T13:29:29.904461+0000", 2026-02-19T13:29:30.175 INFO:tasks.workunit.client.0.trial144.stdout: "schedule": "queued for deep scrub at 2026-02-15T03:04:24.626", 2026-02-19T13:29:30.176 INFO:tasks.workunit.client.0.trial144.stdout: "test_sequence": 7561216 2026-02-19T13:29:30.176 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:30.176 INFO:tasks.workunit.client.0.trial144.stdout: "agent_state": {} 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:( 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[query_epoch]=20 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[query_seq]=46 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[query_active]=false 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule]='queued for deep scrub at 2026-02-15T03:04:24.626' 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule_at]='0' 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[query_target_at]='0' 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_duration]=0 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_stamp]='2026-02-19T13:29:09.366465+0000' 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_scrub]='0x0' 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[query_is_future]=false 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[query_vs_date]=false 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[query_scrub_seq]=7561216 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_pg_state]='active+clean' 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_state_has_scrubbing]=false 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_last_duration]=0 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule]='queued for deep scrub' 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule_at]='0' 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_is_future]=false 2026-02-19T13:29:30.262 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_vs_date]=false 2026-02-19T13:29:30.263 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_reported_epoch]=20 2026-02-19T13:29:30.263 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_seq]=45 2026-02-19T13:29:30.263 INFO:tasks.workunit.client.0.trial144.stdout:) 2026-02-19T13:29:30.272 INFO:tasks.workunit.client.0.trial144.stdout:test counter @ should show no change: 7561216 2026-02-19T13:29:30.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:511: TEST_just_deep_scrubs: echo 'test counter @ should show no change: ' 7561216 2026-02-19T13:29:30.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:512: TEST_just_deep_scrubs: (( 0 == 0 )) 2026-02-19T13:29:30.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:513: TEST_just_deep_scrubs: (( 7561216 == 7561216 )) 2026-02-19T13:29:30.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:516: TEST_just_deep_scrubs: ceph osd unset nodeep-scrub 2026-02-19T13:29:30.475 INFO:tasks.workunit.client.0.trial144.stderr:nodeep-scrub is unset 2026-02-19T13:29:30.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:517: TEST_just_deep_scrubs: sleep 8 2026-02-19T13:29:38.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:518: TEST_just_deep_scrubs: expct_qry_duration=(['query_last_duration']='0' ['query_last_duration_neg']='not0') 2026-02-19T13:29:38.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:518: TEST_just_deep_scrubs: declare -A expct_qry_duration 2026-02-19T13:29:38.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:519: TEST_just_deep_scrubs: sc_data_2=() 2026-02-19T13:29:38.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:520: TEST_just_deep_scrubs: extract_published_sch 1.2 2026-02-19T13:29:24,086336990+00:00 2026-02-19T13:29:24,086336990+00:00 sc_data_2 2026-02-19T13:29:38.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:17: extract_published_sch: local pgn=1.2 2026-02-19T13:29:38.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:18: extract_published_sch: local -n dict=sc_data_2 2026-02-19T13:29:38.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:19: extract_published_sch: local current_time=2026-02-19T13:29:24,086336990+00:00 2026-02-19T13:29:38.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:20: extract_published_sch: local extra_time=2026-02-19T13:29:24,086336990+00:00 2026-02-19T13:29:38.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:21: extract_published_sch: local extr_dbg=2 2026-02-19T13:29:38.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:24: extract_published_sch: local saved_echo_flag=x 2026-02-19T13:29:38.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:25: extract_published_sch: set +x 2026-02-19T13:29:38.634 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:29:38.644 INFO:tasks.workunit.client.0.trial144.stdout:from pg dump pg: { 2026-02-19T13:29:38.644 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_pg_state": "active+clean", 2026-02-19T13:29:38.644 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_state_has_scrubbing": false, 2026-02-19T13:29:38.644 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_last_duration": 1, 2026-02-19T13:29:38.644 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-19T13:29:38.644 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule_at": "2026-02-20T13:29:30.863581+0000", 2026-02-19T13:29:38.644 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_is_future": true, 2026-02-19T13:29:38.644 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_vs_date": true, 2026-02-19T13:29:38.644 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_reported_epoch": 21, 2026-02-19T13:29:38.644 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_seq": 58 2026-02-19T13:29:38.644 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:29:38.644 INFO:tasks.workunit.client.0.trial144.stdout:query output: 2026-02-19T13:29:38.718 INFO:tasks.workunit.client.0.trial144.stdout: "scrubber": { 2026-02-19T13:29:38.718 INFO:tasks.workunit.client.0.trial144.stdout: "active": false, 2026-02-19T13:29:38.718 INFO:tasks.workunit.client.0.trial144.stdout: "must_scrub": false, 2026-02-19T13:29:38.718 INFO:tasks.workunit.client.0.trial144.stdout: "must_deep_scrub": false, 2026-02-19T13:29:38.718 INFO:tasks.workunit.client.0.trial144.stdout: "must_repair": false, 2026-02-19T13:29:38.718 INFO:tasks.workunit.client.0.trial144.stdout: "scrub_reg_stamp": "2026-02-20T13:29:30.863581+0000", 2026-02-19T13:29:38.718 INFO:tasks.workunit.client.0.trial144.stdout: "schedule": "scrub scheduled @ 2026-02-20T13:29:30.863 (2026-02-20T13:29:30.863)", 2026-02-19T13:29:38.718 INFO:tasks.workunit.client.0.trial144.stdout: "test_sequence": 7561219 2026-02-19T13:29:38.718 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:38.718 INFO:tasks.workunit.client.0.trial144.stdout: "agent_state": {} 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:( 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[query_epoch]=21 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[query_seq]=59 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[query_active]=false 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule]='scrub scheduled' 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule_at]='2026-02-20T13:29:30.863' 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[query_target_at]='2026-02-20T13:29:30.863' 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_duration]=1 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_stamp]='2026-02-19T13:29:30.863581+0000' 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_scrub]='18x20' 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[query_is_future]=true 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[query_vs_date]=true 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[query_scrub_seq]=7561219 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_pg_state]='active+clean' 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_state_has_scrubbing]=false 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_last_duration]=1 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule_at]='2026-02-20T13:29:30.863581+0000' 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_is_future]=true 2026-02-19T13:29:38.803 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_vs_date]=true 2026-02-19T13:29:38.804 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_reported_epoch]=21 2026-02-19T13:29:38.804 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_seq]=58 2026-02-19T13:29:38.804 INFO:tasks.workunit.client.0.trial144.stdout:) 2026-02-19T13:29:38.814 INFO:tasks.workunit.client.0.trial144.stdout:test counter @ should be higher than before the unset: 7561219 2026-02-19T13:29:38.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:521: TEST_just_deep_scrubs: echo 'test counter @ should be higher than before the unset: ' 7561219 2026-02-19T13:29:38.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:523: TEST_just_deep_scrubs: expct_qry_duration=(['query_last_duration']='0' ['query_last_duration_neg']='not0') 2026-02-19T13:29:38.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:523: TEST_just_deep_scrubs: declare -A expct_qry_duration 2026-02-19T13:29:38.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:524: TEST_just_deep_scrubs: sc_data_2=() 2026-02-19T13:29:38.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:525: TEST_just_deep_scrubs: wait_any_cond 1.2 10 2026-02-19T13:29:09.366465+0000 expct_qry_duration 'WaitingAfterScrub ' sc_data_2 2026-02-19T13:29:38.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.2 2026-02-19T13:29:38.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-19T13:29:38.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-19T13:29:09.366465+0000 2026-02-19T13:29:38.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=expct_qry_duration 2026-02-19T13:29:38.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sc_data_2 2026-02-19T13:29:38.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-19T13:29:38.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-19T13:29:38.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-19T13:29:38.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-19T13:29:38.815 INFO:tasks.workunit.client.0.trial144.stdout:waiting for any condition (WaitingAfterScrub ): pg:1.2 dt:2026-02-19T13:29:09.366465+0000 (10 retries) 2026-02-19T13:29:39.466 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:29:39.478 INFO:tasks.workunit.client.0.trial144.stdout:from pg dump pg: { 2026-02-19T13:29:39.478 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_pg_state": "active+clean", 2026-02-19T13:29:39.478 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_state_has_scrubbing": false, 2026-02-19T13:29:39.479 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_last_duration": 1, 2026-02-19T13:29:39.479 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-19T13:29:39.479 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_schedule_at": "2026-02-20T13:29:30.863581+0000", 2026-02-19T13:29:39.479 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_is_future": true, 2026-02-19T13:29:39.479 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_vs_date": true, 2026-02-19T13:29:39.479 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_reported_epoch": 21, 2026-02-19T13:29:39.479 INFO:tasks.workunit.client.0.trial144.stdout: "dmp_seq": 58 2026-02-19T13:29:39.479 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:29:39.479 INFO:tasks.workunit.client.0.trial144.stdout:query output: 2026-02-19T13:29:39.553 INFO:tasks.workunit.client.0.trial144.stdout: "scrubber": { 2026-02-19T13:29:39.553 INFO:tasks.workunit.client.0.trial144.stdout: "active": false, 2026-02-19T13:29:39.553 INFO:tasks.workunit.client.0.trial144.stdout: "must_scrub": false, 2026-02-19T13:29:39.553 INFO:tasks.workunit.client.0.trial144.stdout: "must_deep_scrub": false, 2026-02-19T13:29:39.553 INFO:tasks.workunit.client.0.trial144.stdout: "must_repair": false, 2026-02-19T13:29:39.553 INFO:tasks.workunit.client.0.trial144.stdout: "scrub_reg_stamp": "2026-02-20T13:29:30.863581+0000", 2026-02-19T13:29:39.553 INFO:tasks.workunit.client.0.trial144.stdout: "schedule": "scrub scheduled @ 2026-02-20T13:29:30.863 (2026-02-20T13:29:30.863)", 2026-02-19T13:29:39.553 INFO:tasks.workunit.client.0.trial144.stdout: "test_sequence": 7561219 2026-02-19T13:29:39.554 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:39.554 INFO:tasks.workunit.client.0.trial144.stdout: "agent_state": {} 2026-02-19T13:29:39.636 INFO:tasks.workunit.client.0.trial144.stdout:( 2026-02-19T13:29:39.636 INFO:tasks.workunit.client.0.trial144.stdout:[query_epoch]=21 2026-02-19T13:29:39.636 INFO:tasks.workunit.client.0.trial144.stdout:[query_seq]=59 2026-02-19T13:29:39.636 INFO:tasks.workunit.client.0.trial144.stdout:[query_active]=false 2026-02-19T13:29:39.636 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule]='scrub scheduled' 2026-02-19T13:29:39.636 INFO:tasks.workunit.client.0.trial144.stdout:[query_schedule_at]='2026-02-20T13:29:30.863' 2026-02-19T13:29:39.637 INFO:tasks.workunit.client.0.trial144.stdout:[query_target_at]='2026-02-20T13:29:30.863' 2026-02-19T13:29:39.637 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_duration]=1 2026-02-19T13:29:39.637 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_stamp]='2026-02-19T13:29:30.863581+0000' 2026-02-19T13:29:39.637 INFO:tasks.workunit.client.0.trial144.stdout:[query_last_scrub]='18x20' 2026-02-19T13:29:39.637 INFO:tasks.workunit.client.0.trial144.stdout:[query_is_future]=true 2026-02-19T13:29:39.637 INFO:tasks.workunit.client.0.trial144.stdout:[query_vs_date]=true 2026-02-19T13:29:39.637 INFO:tasks.workunit.client.0.trial144.stdout:[query_scrub_seq]=7561219 2026-02-19T13:29:39.637 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_pg_state]='active+clean' 2026-02-19T13:29:39.637 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_state_has_scrubbing]=false 2026-02-19T13:29:39.637 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_last_duration]=1 2026-02-19T13:29:39.637 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-19T13:29:39.637 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_schedule_at]='2026-02-20T13:29:30.863581+0000' 2026-02-19T13:29:39.637 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_is_future]=true 2026-02-19T13:29:39.638 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_vs_date]=true 2026-02-19T13:29:39.638 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_reported_epoch]=21 2026-02-19T13:29:39.638 INFO:tasks.workunit.client.0.trial144.stdout:[dmp_seq]=58 2026-02-19T13:29:39.638 INFO:tasks.workunit.client.0.trial144.stdout:) 2026-02-19T13:29:39.647 INFO:tasks.workunit.client.0.trial144.stdout:--> loop: 1 ~ false / 59 / 58 / true / 2026-02-19T13:29:30.863581+0000 / scrub scheduled %%% query_last_duration_neg query_last_duration 2026-02-19T13:29:39.647 INFO:tasks.workunit.client.0.trial144.stdout:key is query_last_duration: negation:1 # expected: 0 # in actual: 1 2026-02-19T13:29:39.648 INFO:tasks.workunit.client.0.trial144.stdout:WaitingAfterScrub - 'query_last_duration' actual value (1) matches expected (0) (negation: 1) 2026-02-19T13:29:39.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-19T13:29:39.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:526: TEST_just_deep_scrubs: perf_counters td/osd-scrub-test 3 2026-02-19T13:29:39.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-19T13:29:39.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-19T13:29:39.649 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-19T13:29:39.649 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-19T13:29:39.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:29:39.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-19T13:29:39.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:39.729 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 2, 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:29:39.730 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:29:39.731 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:29:39.731 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:29:39.731 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.731 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.731 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:29:39.731 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:29:39.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:29:39.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-19T13:29:39.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:29:39.800 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:29:39.800 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:29:39.800 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:29:39.800 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:29:39.800 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:29:39.801 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:29:39.801 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:39.801 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:29:39.801 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:29:39.801 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:29:39.801 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:29:39.801 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:29:39.801 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:29:39.801 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.801 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.801 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:29:39.801 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:29:39.802 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:29:39.802 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:29:39.802 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:29:39.802 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:29:39.802 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:39.802 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:29:39.802 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:29:39.802 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:29:39.802 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:29:39.802 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:29:39.802 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:29:39.802 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.803 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.803 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:29:39.803 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:29:39.803 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:29:39.803 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:29:39.803 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:29:39.803 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:29:39.803 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:39.803 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:29:39.803 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:29:39.803 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:29:39.803 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:29:39.803 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:29:39.804 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:29:39.804 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.804 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.804 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:29:39.804 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:29:39.804 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:29:39.804 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:29:39.804 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:29:39.804 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:29:39.804 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:39.804 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:29:39.804 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:29:39.804 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:29:39.805 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:29:39.805 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:29:39.805 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:29:39.805 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.805 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.805 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:29:39.805 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:29:39.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:29:39.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-19T13:29:39.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:29:39.872 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:29:39.872 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:29:39.873 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:29:39.873 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:29:39.873 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:29:39.873 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:29:39.873 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:39.873 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:29:39.873 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:29:39.873 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:29:39.873 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:29:39.873 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:29:39.873 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:29:39.873 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.874 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.874 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:29:39.874 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:29:39.874 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:29:39.874 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:29:39.874 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:29:39.874 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:29:39.874 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:39.874 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:29:39.874 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:29:39.874 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:29:39.874 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:29:39.875 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:29:39.875 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:29:39.875 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.875 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.875 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:29:39.875 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:29:39.875 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:29:39.875 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:29:39.875 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:29:39.875 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:29:39.875 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:39.875 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:29:39.875 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:29:39.876 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:29:39.876 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:29:39.876 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:29:39.876 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:29:39.876 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.876 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.876 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:29:39.876 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:29:39.876 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:29:39.876 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:29:39.876 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:29:39.876 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:29:39.876 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:29:39.877 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:29:39.877 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:29:39.877 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:29:39.877 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:29:39.877 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:29:39.877 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:29:39.877 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.877 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:29:39.877 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:29:39.877 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:29:39.878 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Teardown Test TEST_just_deep_scrubs ------------------ 2026-02-19T13:29:39.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_just_deep_scrubs ------------------' 2026-02-19T13:29:39.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-19T13:29:39.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:29:39.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:29:39.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:29:39.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:29:39.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:29:39.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:29:39.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:29:39.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:29:39.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:29:39.983 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:29:39.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:29:39.984 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:29:39.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:29:39.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:29:39.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:29:39.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:29:39.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:29:39.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:29:39.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:29:39.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:29:39.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:29:39.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:29:40.006 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:29:40.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:29:40.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:29:40.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:29:40.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:29:40.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:29:40.008 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Complete Test TEST_just_deep_scrubs ------------------ 2026-02-19T13:29:40.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_just_deep_scrubs ------------------' 2026-02-19T13:29:40.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-19T13:29:40.009 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Prepare Test TEST_pg_dump_objects_scrubbed ------------------- 2026-02-19T13:29:40.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_pg_dump_objects_scrubbed -------------------' 2026-02-19T13:29:40.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-19T13:29:40.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-test 2026-02-19T13:29:40.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-test 2026-02-19T13:29:40.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:29:40.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:29:40.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:29:40.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:29:40.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:29:40.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:29:40.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:29:40.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:29:40.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:29:40.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:29:40.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:29:40.013 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:29:40.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:29:40.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:29:40.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:29:40.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:29:40.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:29:40.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:29:40.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:29:40.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:29:40.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:29:40.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:29:40.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:29:40.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:29:40.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:29:40.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:29:40.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:29:40.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:29:40.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-test 2026-02-19T13:29:40.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:29:40.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:29:40.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:29:40.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.511299 2026-02-19T13:29:40.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:29:40.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:29:40.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:29:40.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-19T13:29:40.025 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Run Test TEST_pg_dump_objects_scrubbed ----------------------- 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_pg_dump_objects_scrubbed -----------------------' 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_pg_dump_objects_scrubbed td/osd-scrub-test 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:653: TEST_pg_dump_objects_scrubbed: local dir=td/osd-scrub-test 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:654: TEST_pg_dump_objects_scrubbed: local poolname=test 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:655: TEST_pg_dump_objects_scrubbed: local OSDS=3 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:656: TEST_pg_dump_objects_scrubbed: local objects=15 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:657: TEST_pg_dump_objects_scrubbed: local timeout=10 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:659: TEST_pg_dump_objects_scrubbed: TESTDATA=testdata.511299 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:661: TEST_pg_dump_objects_scrubbed: setup td/osd-scrub-test 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-test 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-test 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:29:40.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:29:40.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:29:40.029 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:29:40.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:29:40.030 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:29:40.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:29:40.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:29:40.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:29:40.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:29:40.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:29:40.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:29:40.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:29:40.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:29:40.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:29:40.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:29:40.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:29:40.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:29:40.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:29:40.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:29:40.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:29:40.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:29:40.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-test 2026-02-19T13:29:40.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:29:40.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:29:40.040 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:29:40.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.511299 2026-02-19T13:29:40.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:29:40.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:29:40.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:29:40.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-19T13:29:40.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:662: TEST_pg_dump_objects_scrubbed: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-19T13:29:40.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-test 2026-02-19T13:29:40.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:29:40.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:29:40.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:29:40.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-test/a 2026-02-19T13:29:40.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-19T13:29:40.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:29:40.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:29:40.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:29:40.066 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:29:40.066 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:29:40.066 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:29:40.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:29:40.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-19T13:29:40.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:29:40.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:29:40.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:29:40.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:29:40.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:29:40.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:29:40.096 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:29:40.096 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:29:40.096 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:29:40.097 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:29:40.097 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:29:40.097 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:29:40.097 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-mon.a.asok 2026-02-19T13:29:40.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:29:40.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.511299/ceph-mon.a.asok config get fsid 2026-02-19T13:29:40.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:29:40.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:29:40.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:29:40.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:29:40.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:29:40.153 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:29:40.153 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:29:40.153 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:29:40.154 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:29:40.154 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:29:40.154 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:29:40.154 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-mon.a.asok 2026-02-19T13:29:40.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:29:40.154 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.511299/ceph-mon.a.asok config get mon_host 2026-02-19T13:29:40.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:663: TEST_pg_dump_objects_scrubbed: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-19T13:29:40.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-test 2026-02-19T13:29:40.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:29:40.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:29:40.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:29:40.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-test/x 2026-02-19T13:29:40.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:29:40.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:29:40.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:29:40.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:29:40.306 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:29:40.306 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:29:40.306 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:29:40.306 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:29:40.307 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:29:40.307 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-19T13:29:40.326 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:664: TEST_pg_dump_objects_scrubbed: expr 3 - 1 2026-02-19T13:29:40.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:664: TEST_pg_dump_objects_scrubbed: seq 0 2 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:664: TEST_pg_dump_objects_scrubbed: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:666: TEST_pg_dump_objects_scrubbed: run_osd td/osd-scrub-test 0 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:29:40.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:29:40.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:29:40.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:29:40.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-19T13:29:40.331 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:29:40.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d7c3b031-1377-4b71-a0ed-cf09bf6a6271 2026-02-19T13:29:40.332 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 d7c3b031-1377-4b71-a0ed-cf09bf6a6271 2026-02-19T13:29:40.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d7c3b031-1377-4b71-a0ed-cf09bf6a6271' 2026-02-19T13:29:40.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:29:40.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBEEJdpnBBmFBAAc5Zb7WdeMjMp458dxTZRlQ== 2026-02-19T13:29:40.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBEEJdpnBBmFBAAc5Zb7WdeMjMp458dxTZRlQ=="}' 2026-02-19T13:29:40.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d7c3b031-1377-4b71-a0ed-cf09bf6a6271 -i td/osd-scrub-test/0/new.json 2026-02-19T13:29:40.428 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:29:40.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-19T13:29:40.437 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBEEJdpnBBmFBAAc5Zb7WdeMjMp458dxTZRlQ== --osd-uuid d7c3b031-1377-4b71-a0ed-cf09bf6a6271 2026-02-19T13:29:40.452 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:40.451+0000 7f6a41a79a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:40.453 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:40.451+0000 7f6a41a79a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:40.454 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:40.451+0000 7f6a41a79a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:40.454 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:40.451+0000 7f6a41a79a40 -1 bdev(0x564c99162c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:29:40.454 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:40.451+0000 7f6a41a79a40 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-19T13:29:41.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-19T13:29:41.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:29:41.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:29:41.240 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:29:41.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:29:41.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:29:41.442 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:29:41.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:29:41.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:29:41.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:29:41.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:29:41.467 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:41.467+0000 7fc398276a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:41.469 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:41.467+0000 7fc398276a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:41.470 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:41.467+0000 7fc398276a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:41.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:29:41.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:29:41.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:29:41.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:29:41.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:29:41.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:29:41.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:29:41.599 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:29:41.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:29:41.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:29:41.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:29:41.852 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:41.851+0000 7fc398276a40 -1 Falling back to public interface 2026-02-19T13:29:42.417 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:42.415+0000 7fc398276a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:29:42.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:29:42.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:29:42.763 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:29:42.764 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:29:42.764 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:29:42.764 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:29:42.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:29:43.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:29:43.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:29:43.925 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:29:43.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:29:43.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:29:43.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:29:44.086 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2533610322,v1:127.0.0.1:6803/2533610322] [v2:127.0.0.1:6804/2533610322,v1:127.0.0.1:6805/2533610322] exists,up d7c3b031-1377-4b71-a0ed-cf09bf6a6271 2026-02-19T13:29:44.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:664: TEST_pg_dump_objects_scrubbed: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:666: TEST_pg_dump_objects_scrubbed: run_osd td/osd-scrub-test 1 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:29:44.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:29:44.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-19T13:29:44.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:29:44.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=58237b0e-b371-4e02-8b78-a9bf789874d5 2026-02-19T13:29:44.091 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 58237b0e-b371-4e02-8b78-a9bf789874d5 2026-02-19T13:29:44.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 58237b0e-b371-4e02-8b78-a9bf789874d5' 2026-02-19T13:29:44.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:29:44.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBIEJdpLiRABhAAUhPP6Pr9d0/yNbwWxd2DvQ== 2026-02-19T13:29:44.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBIEJdpLiRABhAAUhPP6Pr9d0/yNbwWxd2DvQ=="}' 2026-02-19T13:29:44.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 58237b0e-b371-4e02-8b78-a9bf789874d5 -i td/osd-scrub-test/1/new.json 2026-02-19T13:29:44.266 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:29:44.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-19T13:29:44.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBIEJdpLiRABhAAUhPP6Pr9d0/yNbwWxd2DvQ== --osd-uuid 58237b0e-b371-4e02-8b78-a9bf789874d5 2026-02-19T13:29:44.292 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:44.290+0000 7f96a8ea8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:44.294 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:44.290+0000 7f96a8ea8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:44.295 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:44.294+0000 7f96a8ea8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:44.295 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:44.294+0000 7f96a8ea8a40 -1 bdev(0x55b78440ac00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:29:44.296 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:44.294+0000 7f96a8ea8a40 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-19T13:29:44.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-19T13:29:44.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:29:44.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:29:44.961 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:29:44.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:29:45.164 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:29:45.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:29:45.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:29:45.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:29:45.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:29:45.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:29:45.188 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:45.186+0000 7f3616679a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:45.189 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:45.186+0000 7f3616679a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:45.190 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:45.186+0000 7f3616679a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:45.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:29:45.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:29:45.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:29:45.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:29:45.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:29:45.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:29:45.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:29:45.322 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:29:45.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:29:45.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:29:45.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:29:45.584 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:45.582+0000 7f3616679a40 -1 Falling back to public interface 2026-02-19T13:29:46.140 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:46.138+0000 7f3616679a40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:29:46.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:29:46.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:29:46.480 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:29:46.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:29:46.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:29:46.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:29:46.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:29:47.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:29:47.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:29:47.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:29:47.646 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:29:47.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:29:47.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:29:47.800 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4168127011,v1:127.0.0.1:6811/4168127011] [v2:127.0.0.1:6812/4168127011,v1:127.0.0.1:6813/4168127011] exists,up 58237b0e-b371-4e02-8b78-a9bf789874d5 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:664: TEST_pg_dump_objects_scrubbed: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:666: TEST_pg_dump_objects_scrubbed: run_osd td/osd-scrub-test 2 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-19T13:29:47.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:29:47.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:29:47.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:29:47.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:29:47.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:29:47.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:29:47.802 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:29:47.802 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:29:47.802 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:29:47.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:29:47.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:29:47.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:29:47.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:29:47.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:29:47.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:29:47.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:29:47.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:29:47.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:29:47.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:29:47.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:29:47.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:29:47.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:29:47.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-19T13:29:47.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:29:47.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e9f26865-de8d-4c6a-806f-e4150e837c73 2026-02-19T13:29:47.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 e9f26865-de8d-4c6a-806f-e4150e837c73' 2026-02-19T13:29:47.805 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 e9f26865-de8d-4c6a-806f-e4150e837c73 2026-02-19T13:29:47.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:29:47.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBLEJdp73CtMBAAH1mPokTzGhKzwr2kcA1Muw== 2026-02-19T13:29:47.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBLEJdp73CtMBAAH1mPokTzGhKzwr2kcA1Muw=="}' 2026-02-19T13:29:47.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e9f26865-de8d-4c6a-806f-e4150e837c73 -i td/osd-scrub-test/2/new.json 2026-02-19T13:29:47.977 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:29:47.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-19T13:29:47.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBLEJdp73CtMBAAH1mPokTzGhKzwr2kcA1Muw== --osd-uuid e9f26865-de8d-4c6a-806f-e4150e837c73 2026-02-19T13:29:48.003 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:48.002+0000 7f805a502a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:48.005 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:48.002+0000 7f805a502a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:48.006 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:48.002+0000 7f805a502a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:48.006 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:48.002+0000 7f805a502a40 -1 bdev(0x55886f620c00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:29:48.006 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:48.002+0000 7f805a502a40 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-19T13:29:48.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-19T13:29:48.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:29:48.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:29:48.679 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T13:29:48.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:29:48.874 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T13:29:48.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:29:48.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:29:48.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:29:48.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:29:48.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:29:48.898 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:48.894+0000 7fa5c849ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:48.900 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:48.898+0000 7fa5c849ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:48.901 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:48.898+0000 7fa5c849ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:29:49.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:29:49.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:29:49.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:29:49.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:29:49.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:29:49.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:29:49.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:29:49.033 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:29:49.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:29:49.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:29:49.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:29:49.228 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:49.226+0000 7fa5c849ba40 -1 Falling back to public interface 2026-02-19T13:29:49.820 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:29:49.818+0000 7fa5c849ba40 -1 osd.2 0 log_to_monitors true 2026-02-19T13:29:50.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:29:50.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:29:50.192 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:29:50.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:29:50.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:29:50.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:29:50.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:29:51.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:29:51.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:29:51.352 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:29:51.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:29:51.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:29:51.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:29:51.510 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3567808165,v1:127.0.0.1:6819/3567808165] [v2:127.0.0.1:6820/3567808165,v1:127.0.0.1:6821/3567808165] exists,up e9f26865-de8d-4c6a-806f-e4150e837c73 2026-02-19T13:29:51.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:29:51.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:29:51.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:29:51.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:670: TEST_pg_dump_objects_scrubbed: create_pool test 1 1 2026-02-19T13:29:51.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-19T13:29:51.716 INFO:tasks.workunit.client.0.trial144.stderr:pool 'test' created 2026-02-19T13:29:51.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:29:52.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:671: TEST_pg_dump_objects_scrubbed: wait_for_clean 2026-02-19T13:29:52.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:29:52.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:29:52.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:29:52.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:29:52.730 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:29:52.730 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:29:52.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:29:52.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:29:52.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:29:52.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:29:52.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:29:52.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:29:52.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:29:52.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:29:52.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:29:53.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:29:53.002 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:29:53.002 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:29:53.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:29:53.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:29:53.003 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:29:53.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-19T13:29:53.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-19T13:29:53.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491' 2026-02-19T13:29:53.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:29:53.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:29:53.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705671 2026-02-19T13:29:53.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705671 2026-02-19T13:29:53.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705671' 2026-02-19T13:29:53.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:29:53.145 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:29:53.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542148 2026-02-19T13:29:53.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542148 2026-02-19T13:29:53.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705671 2-60129542148' 2026-02-19T13:29:53.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:29:53.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-19T13:29:53.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:29:53.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:29:53.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836491 2026-02-19T13:29:53.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:29:53.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836491 2026-02-19T13:29:53.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-19T13:29:53.221 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836491 2026-02-19T13:29:53.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:29:53.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T13:29:53.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:29:54.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:29:54.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:29:54.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T13:29:54.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:29:55.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-19T13:29:55.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:29:55.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836493 -lt 21474836491 2026-02-19T13:29:55.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:29:55.704 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705671 2026-02-19T13:29:55.704 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:29:55.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:29:55.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705671 2026-02-19T13:29:55.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:29:55.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705671 2026-02-19T13:29:55.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-19T13:29:55.708 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705671 2026-02-19T13:29:55.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:29:55.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705673 -lt 38654705671 2026-02-19T13:29:55.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:29:55.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542148 2026-02-19T13:29:55.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:29:55.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:29:55.873 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542148 2026-02-19T13:29:55.873 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:29:55.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542148 2026-02-19T13:29:55.874 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542148 2026-02-19T13:29:55.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-19T13:29:55.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:29:56.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542149 -lt 60129542148 2026-02-19T13:29:56.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:29:56.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:29:56.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:29:56.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:29:56.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:29:56.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:29:56.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:29:56.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:29:56.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:29:56.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:29:56.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:29:56.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:29:56.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:29:56.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:29:56.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:29:56.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:29:56.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:29:56.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:29:56.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:672: TEST_pg_dump_objects_scrubbed: ceph osd dump 2026-02-19T13:29:56.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:672: TEST_pg_dump_objects_scrubbed: grep '^pool.*['\'']test['\'']' 2026-02-19T13:29:56.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:672: TEST_pg_dump_objects_scrubbed: awk '{ print $2 }' 2026-02-19T13:29:56.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:672: TEST_pg_dump_objects_scrubbed: poolid=1 2026-02-19T13:29:56.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:674: TEST_pg_dump_objects_scrubbed: dd if=/dev/urandom of=testdata.511299 bs=1032 count=1 2026-02-19T13:29:56.740 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:29:56.740 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:29:56.740 INFO:tasks.workunit.client.0.trial144.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.788e-05 s, 27.2 MB/s 2026-02-19T13:29:56.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: seq 1 15 2026-02-19T13:29:56.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-19T13:29:56.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rados -p test put obj1 testdata.511299 2026-02-19T13:29:56.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-19T13:29:56.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rados -p test put obj2 testdata.511299 2026-02-19T13:29:56.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-19T13:29:56.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rados -p test put obj3 testdata.511299 2026-02-19T13:29:56.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-19T13:29:56.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rados -p test put obj4 testdata.511299 2026-02-19T13:29:56.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-19T13:29:56.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rados -p test put obj5 testdata.511299 2026-02-19T13:29:56.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-19T13:29:56.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rados -p test put obj6 testdata.511299 2026-02-19T13:29:56.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-19T13:29:56.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rados -p test put obj7 testdata.511299 2026-02-19T13:29:56.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-19T13:29:56.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rados -p test put obj8 testdata.511299 2026-02-19T13:29:56.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-19T13:29:56.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rados -p test put obj9 testdata.511299 2026-02-19T13:29:56.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-19T13:29:56.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rados -p test put obj10 testdata.511299 2026-02-19T13:29:56.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-19T13:29:56.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rados -p test put obj11 testdata.511299 2026-02-19T13:29:56.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-19T13:29:56.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rados -p test put obj12 testdata.511299 2026-02-19T13:29:57.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-19T13:29:57.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rados -p test put obj13 testdata.511299 2026-02-19T13:29:57.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-19T13:29:57.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rados -p test put obj14 testdata.511299 2026-02-19T13:29:57.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-19T13:29:57.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rados -p test put obj15 testdata.511299 2026-02-19T13:29:57.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:679: TEST_pg_dump_objects_scrubbed: rm -f testdata.511299 2026-02-19T13:29:57.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:681: TEST_pg_dump_objects_scrubbed: local pgid=1.0 2026-02-19T13:29:57.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:683: TEST_pg_dump_objects_scrubbed: pg_scrub 1.0 2026-02-19T13:29:57.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-19T13:29:57.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-19T13:29:57.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-19T13:29:57.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:29:57.084 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:29:57.084 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:29:57.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:29:57.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:29:57.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:29:57.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:29:57.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:29:57.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:29:57.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:29:57.376 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:29:57.376 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:29:57.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:29:57.531 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:29:57.531 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:29:57.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:29:57.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:29:57.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:29:57.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836497 2026-02-19T13:29:57.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836497 2026-02-19T13:29:57.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497' 2026-02-19T13:29:57.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:29:57.601 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:29:57.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705677 2026-02-19T13:29:57.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705677 2026-02-19T13:29:57.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-38654705677' 2026-02-19T13:29:57.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:29:57.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:29:57.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542153 2026-02-19T13:29:57.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542153 2026-02-19T13:29:57.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836497 1-38654705677 2-60129542153' 2026-02-19T13:29:57.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:29:57.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836497 2026-02-19T13:29:57.745 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:29:57.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:29:57.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836497 2026-02-19T13:29:57.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:29:57.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836497 2026-02-19T13:29:57.749 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836497 2026-02-19T13:29:57.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2026-02-19T13:29:57.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:29:57.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836495 -lt 21474836497 2026-02-19T13:29:57.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:29:58.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:29:58.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:29:59.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836498 -lt 21474836497 2026-02-19T13:29:59.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:29:59.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705677 2026-02-19T13:29:59.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:29:59.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:29:59.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705677 2026-02-19T13:29:59.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:29:59.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705677 2026-02-19T13:29:59.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705677' 2026-02-19T13:29:59.077 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705677 2026-02-19T13:29:59.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:29:59.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705678 -lt 38654705677 2026-02-19T13:29:59.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:29:59.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542153 2026-02-19T13:29:59.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:29:59.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:29:59.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542153 2026-02-19T13:29:59.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:29:59.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542153 2026-02-19T13:29:59.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-19T13:29:59.245 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542153 2026-02-19T13:29:59.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:29:59.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542154 -lt 60129542153 2026-02-19T13:29:59.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:29:59.415 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 1.0 loop 0 2026-02-19T13:29:59.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-19T13:29:59.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-19T13:29:59.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-19T13:29:59.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:29:59.416 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-19T13:29:59.416 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:29:59.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-19T13:29:59.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:29:59.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:29:59.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:29:59.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 1.0 2026-02-19T13:29:59.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:29:59.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:29:59.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:29:59.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:29:59.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-19T13:29:51.716967+0000 2026-02-19T13:29:59.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 1.0 2026-02-19T13:29:59.790 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.0 on osd.1 to scrub 2026-02-19T13:29:59.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 1.0 2026-02-19T13:29:51.716967+0000 2026-02-19T13:29:59.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T13:29:59.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:29:51.716967+0000 2026-02-19T13:29:59.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:29:59.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:29:59.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:29:59.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:29:59.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:29:59.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:29:59.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:29:59.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:29:59.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:29:51.716967+0000 '>' 2026-02-19T13:29:51.716967+0000 2026-02-19T13:29:59.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:30:00.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:30:00.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:30:00.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:30:00.961 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:30:00.961 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:30:00.961 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:30:00.961 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:30:01.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:30:00.162858+0000 '>' 2026-02-19T13:29:51.716967+0000 2026-02-19T13:30:01.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:30:01.120 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:684: TEST_pg_dump_objects_scrubbed: ceph pg 1.0 query 2026-02-19T13:30:01.120 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:684: TEST_pg_dump_objects_scrubbed: jq .info.stats.objects_scrubbed 2026-02-19T13:30:01.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:684: TEST_pg_dump_objects_scrubbed: test 15 = 15 2026-02-19T13:30:01.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:685: TEST_pg_dump_objects_scrubbed: perf_counters td/osd-scrub-test 3 2026-02-19T13:30:01.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-19T13:30:01.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-19T13:30:01.193 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-19T13:30:01.193 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-19T13:30:01.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:30:01.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-19T13:30:01.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:30:01.265 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:30:01.266 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:30:01.266 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:30:01.266 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:30:01.266 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:30:01.266 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:30:01.266 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:30:01.266 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:30:01.266 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:30:01.267 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:30:01.267 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:30:01.267 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:30:01.267 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:30:01.267 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.267 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.267 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:30:01.267 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:30:01.267 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:30:01.268 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:30:01.268 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:30:01.268 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:30:01.268 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:30:01.268 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:30:01.268 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:30:01.268 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:30:01.268 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:30:01.268 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:30:01.268 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:30:01.268 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.269 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.269 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:30:01.269 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:30:01.269 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:30:01.269 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:30:01.269 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:30:01.269 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:30:01.269 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:30:01.269 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:30:01.269 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:30:01.269 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:30:01.270 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:30:01.270 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:30:01.270 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:30:01.270 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.270 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.270 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:30:01.270 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:30:01.270 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:30:01.270 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:30:01.270 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:30:01.270 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:30:01.271 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:30:01.271 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:30:01.271 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:30:01.271 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:30:01.271 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:30:01.271 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:30:01.271 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:30:01.271 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.271 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.272 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:30:01.272 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:30:01.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:30:01.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-19T13:30:01.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:30:01.339 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:30:01.339 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:30:01.339 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:30:01.339 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:30:01.339 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:30:01.339 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:30:01.339 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:30:01.339 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:30:01.340 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:30:01.340 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:30:01.340 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:30:01.340 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:30:01.340 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:30:01.340 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.340 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.340 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:30:01.340 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:30:01.340 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:30:01.341 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:30:01.341 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:30:01.341 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:30:01.341 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:30:01.341 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:30:01.341 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:30:01.341 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:30:01.341 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:30:01.341 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:30:01.341 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:30:01.342 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.342 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.342 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:30:01.342 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:30:01.342 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:30:01.342 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:30:01.342 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:30:01.342 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:30:01.342 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:30:01.342 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:30:01.342 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:30:01.342 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:30:01.343 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:30:01.343 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:30:01.343 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:30:01.343 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.343 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.343 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:30:01.343 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:30:01.343 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:30:01.343 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:30:01.343 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:30:01.343 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:30:01.343 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:30:01.344 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:30:01.344 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:30:01.344 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 1, 2026-02-19T13:30:01.344 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:30:01.344 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:30:01.344 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:30:01.344 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.344 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.344 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:30:01.344 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:30:01.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:30:01.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-19T13:30:01.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:30:01.408 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:30:01.408 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:30:01.409 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:30:01.410 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:30:01.411 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:30:01.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:687: TEST_pg_dump_objects_scrubbed: teardown td/osd-scrub-test 2026-02-19T13:30:01.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:30:01.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:30:01.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:30:01.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:30:01.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:30:01.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:30:01.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:30:01.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:30:01.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:30:01.520 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:30:01.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:30:01.521 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:30:01.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:30:01.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:30:01.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:30:01.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:30:01.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:30:01.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:30:01.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:30:01.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:30:01.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:30:01.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:30:01.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:30:01.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:30:01.536 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:30:01.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:30:01.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:30:01.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:30:01.538 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Teardown Test TEST_pg_dump_objects_scrubbed ------------------ 2026-02-19T13:30:01.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_pg_dump_objects_scrubbed ------------------' 2026-02-19T13:30:01.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-19T13:30:01.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:30:01.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:30:01.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:30:01.539 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:30:01.540 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:30:01.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:30:01.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:30:01.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:30:01.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:30:01.540 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:30:01.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:30:01.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:30:01.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:30:01.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:30:01.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:30:01.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:30:01.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:30:01.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:30:01.546 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:30:01.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:30:01.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:30:01.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:30:01.550 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:30:01.550 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:30:01.550 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:30:01.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:30:01.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:30:01.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:30:01.552 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Complete Test TEST_pg_dump_objects_scrubbed ------------------ 2026-02-19T13:30:01.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_pg_dump_objects_scrubbed ------------------' 2026-02-19T13:30:01.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-19T13:30:01.552 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Prepare Test TEST_scrub_abort ------------------- 2026-02-19T13:30:01.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_scrub_abort -------------------' 2026-02-19T13:30:01.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-19T13:30:01.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-test 2026-02-19T13:30:01.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-test 2026-02-19T13:30:01.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:30:01.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:30:01.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:30:01.554 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:30:01.554 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:30:01.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:30:01.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:30:01.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:30:01.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:30:01.555 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:30:01.556 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:30:01.556 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:30:01.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:30:01.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:30:01.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:30:01.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:30:01.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:30:01.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:30:01.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:30:01.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:30:01.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:30:01.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:30:01.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:30:01.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:30:01.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:30:01.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:30:01.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:30:01.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:30:01.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-test 2026-02-19T13:30:01.566 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:30:01.566 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:30:01.566 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:30:01.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.511299 2026-02-19T13:30:01.568 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:30:01.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:30:01.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:30:01.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-19T13:30:01.569 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Run Test TEST_scrub_abort ----------------------- 2026-02-19T13:30:01.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_scrub_abort -----------------------' 2026-02-19T13:30:01.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_scrub_abort td/osd-scrub-test 2026-02-19T13:30:01.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:404: TEST_scrub_abort: local dir=td/osd-scrub-test 2026-02-19T13:30:01.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:405: TEST_scrub_abort: _scrub_abort td/osd-scrub-test scrub 2026-02-19T13:30:01.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:293: _scrub_abort: local dir=td/osd-scrub-test 2026-02-19T13:30:01.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:294: _scrub_abort: local poolname=test 2026-02-19T13:30:01.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:295: _scrub_abort: local OSDS=3 2026-02-19T13:30:01.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:296: _scrub_abort: local objects=1000 2026-02-19T13:30:01.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:297: _scrub_abort: local type=scrub 2026-02-19T13:30:01.571 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:299: _scrub_abort: TESTDATA=testdata.511299 2026-02-19T13:30:01.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:300: _scrub_abort: test scrub = scrub 2026-02-19T13:30:01.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:302: _scrub_abort: stopscrub=noscrub 2026-02-19T13:30:01.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:303: _scrub_abort: check=noscrub 2026-02-19T13:30:01.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:309: _scrub_abort: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-19T13:30:01.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-test 2026-02-19T13:30:01.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:30:01.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:30:01.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:30:01.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-test/a 2026-02-19T13:30:01.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-19T13:30:01.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:30:01.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:30:01.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:30:01.591 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:30:01.591 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:30:01.591 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:30:01.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:30:01.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-19T13:30:01.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:30:01.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:30:01.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:30:01.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:30:01.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:30:01.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:30:01.620 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:30:01.620 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:30:01.620 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:30:01.621 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:30:01.621 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:30:01.621 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:30:01.621 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-mon.a.asok 2026-02-19T13:30:01.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:30:01.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.511299/ceph-mon.a.asok config get fsid 2026-02-19T13:30:01.693 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:30:01.693 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:30:01.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:30:01.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:30:01.694 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:30:01.695 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:30:01.695 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:30:01.695 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:30:01.695 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:30:01.696 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:30:01.696 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:30:01.696 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-mon.a.asok 2026-02-19T13:30:01.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:30:01.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.511299/ceph-mon.a.asok config get mon_host 2026-02-19T13:30:01.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:310: _scrub_abort: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-19T13:30:01.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-test 2026-02-19T13:30:01.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:30:01.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:30:01.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:30:01.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-test/x 2026-02-19T13:30:01.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:30:01.841 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:30:01.841 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:30:01.841 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:30:01.841 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:30:01.841 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:30:01.841 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:30:01.841 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:30:01.842 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:30:01.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-19T13:30:01.860 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:311: _scrub_abort: expr 3 - 1 2026-02-19T13:30:01.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:311: _scrub_abort: seq 0 2 2026-02-19T13:30:01.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:311: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:30:01.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:317: _scrub_abort: run_osd td/osd-scrub-test 0 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-19T13:30:01.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:30:01.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:30:01.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:30:01.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:30:01.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-19T13:30:01.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:30:01.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:30:01.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:30:01.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:30:01.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-19T13:30:01.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-19T13:30:01.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:30:01.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:30:01.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:30:01.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:30:01.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:30:01.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:30:01.866 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:30:01.866 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:30:01.866 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:30:01.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:30:01.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:30:01.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:30:01.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:30:01.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:30:01.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:30:01.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:30:01.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:30:01.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:30:01.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:30:01.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:30:01.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:30:01.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-19T13:30:01.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-19T13:30:01.868 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:30:01.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e20951a8-d706-4443-affa-a158843be99f 2026-02-19T13:30:01.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 e20951a8-d706-4443-affa-a158843be99f' 2026-02-19T13:30:01.869 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 e20951a8-d706-4443-affa-a158843be99f 2026-02-19T13:30:01.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:30:01.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBZEJdpJShKNBAAmg+u2QdInpPSHuktRBsdLA== 2026-02-19T13:30:01.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBZEJdpJShKNBAAmg+u2QdInpPSHuktRBsdLA=="}' 2026-02-19T13:30:01.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e20951a8-d706-4443-affa-a158843be99f -i td/osd-scrub-test/0/new.json 2026-02-19T13:30:01.965 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:30:01.973 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-19T13:30:01.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQBZEJdpJShKNBAAmg+u2QdInpPSHuktRBsdLA== --osd-uuid e20951a8-d706-4443-affa-a158843be99f 2026-02-19T13:30:01.991 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:01.986+0000 7f99768f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:01.992 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:01.990+0000 7f99768f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:01.993 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:01.990+0000 7f99768f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:01.993 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:01.990+0000 7f99768f2a40 -1 bdev(0x5561be944c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:30:01.993 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:01.990+0000 7f99768f2a40 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-19T13:30:02.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-19T13:30:02.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:30:02.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:30:02.740 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:30:02.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:30:02.932 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:30:02.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:30:02.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-19T13:30:02.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:30:02.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:30:02.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:30:02.958 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:02.954+0000 7f0f518c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:02.959 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:02.958+0000 7f0f518c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:02.961 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:02.958+0000 7f0f518c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:03.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:30:03.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:30:03.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:30:03.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:30:03.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:30:03.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:30:03.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:30:03.095 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:30:03.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:30:03.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:30:03.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:30:03.324 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:03.322+0000 7f0f518c7a40 -1 Falling back to public interface 2026-02-19T13:30:03.883 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:03.882+0000 7f0f518c7a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:30:04.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:30:04.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:30:04.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:30:04.250 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:30:04.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:30:04.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:30:04.403 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/937459608,v1:127.0.0.1:6803/937459608] [v2:127.0.0.1:6804/937459608,v1:127.0.0.1:6805/937459608] exists,up e20951a8-d706-4443-affa-a158843be99f 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:311: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:317: _scrub_abort: run_osd td/osd-scrub-test 1 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:30:04.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:30:04.405 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:30:04.405 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:30:04.405 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:30:04.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:30:04.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:30:04.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:30:04.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:30:04.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:30:04.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:30:04.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:30:04.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:30:04.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:30:04.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:30:04.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:30:04.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:30:04.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-19T13:30:04.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-19T13:30:04.407 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:30:04.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7ef78792-c678-45fa-93fc-b007869a5e7b 2026-02-19T13:30:04.408 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 7ef78792-c678-45fa-93fc-b007869a5e7b 2026-02-19T13:30:04.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 7ef78792-c678-45fa-93fc-b007869a5e7b' 2026-02-19T13:30:04.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:30:04.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBcEJdpMnEgGRAATipikus1TObWY1UPcDVvYg== 2026-02-19T13:30:04.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBcEJdpMnEgGRAATipikus1TObWY1UPcDVvYg=="}' 2026-02-19T13:30:04.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7ef78792-c678-45fa-93fc-b007869a5e7b -i td/osd-scrub-test/1/new.json 2026-02-19T13:30:04.580 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:30:04.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-19T13:30:04.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQBcEJdpMnEgGRAATipikus1TObWY1UPcDVvYg== --osd-uuid 7ef78792-c678-45fa-93fc-b007869a5e7b 2026-02-19T13:30:04.608 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:04.606+0000 7fae4b0ffa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:04.609 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:04.606+0000 7fae4b0ffa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:04.610 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:04.606+0000 7fae4b0ffa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:04.611 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:04.610+0000 7fae4b0ffa40 -1 bdev(0x56511aa14c00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:30:04.611 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:04.610+0000 7fae4b0ffa40 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-19T13:30:05.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-19T13:30:05.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:30:05.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:30:05.239 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:30:05.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:30:05.439 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:30:05.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:30:05.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-19T13:30:05.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:30:05.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:30:05.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:30:05.464 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:05.462+0000 7f42d6834a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:05.466 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:05.462+0000 7f42d6834a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:05.467 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:05.466+0000 7f42d6834a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:05.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:30:05.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:30:05.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:30:05.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:30:05.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:30:05.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:30:05.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:30:05.603 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:30:05.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:30:05.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:30:05.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:30:05.812 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:05.810+0000 7f42d6834a40 -1 Falling back to public interface 2026-02-19T13:30:06.372 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:06.370+0000 7f42d6834a40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:30:06.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:30:06.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:30:06.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:30:06.763 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:30:06.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:30:06.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:30:06.918 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4032685010,v1:127.0.0.1:6811/4032685010] [v2:127.0.0.1:6812/4032685010,v1:127.0.0.1:6813/4032685010] exists,up 7ef78792-c678-45fa-93fc-b007869a5e7b 2026-02-19T13:30:06.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:30:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:30:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:30:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:311: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:30:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:317: _scrub_abort: run_osd td/osd-scrub-test 2 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-19T13:30:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:30:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:30:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:30:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:30:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-19T13:30:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:30:06.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:30:06.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:30:06.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:30:06.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-19T13:30:06.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-19T13:30:06.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:30:06.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:30:06.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:30:06.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:30:06.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:30:06.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:30:06.923 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:30:06.923 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:30:06.924 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:30:06.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:30:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:30:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:30:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:30:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:30:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:30:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:30:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:30:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:30:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:30:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:30:06.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:30:06.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-19T13:30:06.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-19T13:30:06.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:30:06.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=565a8be8-b2da-46a8-b2b2-b36b1934ba0a 2026-02-19T13:30:06.926 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 565a8be8-b2da-46a8-b2b2-b36b1934ba0a 2026-02-19T13:30:06.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 565a8be8-b2da-46a8-b2b2-b36b1934ba0a' 2026-02-19T13:30:06.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:30:06.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBeEJdp/Cy8NxAAmUpEfuatcOVmnKlSh0APVg== 2026-02-19T13:30:06.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBeEJdp/Cy8NxAAmUpEfuatcOVmnKlSh0APVg=="}' 2026-02-19T13:30:06.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 565a8be8-b2da-46a8-b2b2-b36b1934ba0a -i td/osd-scrub-test/2/new.json 2026-02-19T13:30:07.084 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:30:07.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-19T13:30:07.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQBeEJdp/Cy8NxAAmUpEfuatcOVmnKlSh0APVg== --osd-uuid 565a8be8-b2da-46a8-b2b2-b36b1934ba0a 2026-02-19T13:30:07.112 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:07.110+0000 7f93abccfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:07.113 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:07.110+0000 7f93abccfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:07.114 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:07.110+0000 7f93abccfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:07.115 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:07.114+0000 7f93abccfa40 -1 bdev(0x55883da92c00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:30:07.115 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:07.114+0000 7f93abccfa40 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-19T13:30:07.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-19T13:30:07.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:30:07.770 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T13:30:07.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:30:07.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:30:07.960 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T13:30:07.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:30:07.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-19T13:30:07.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:30:07.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:30:07.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:30:07.986 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:07.982+0000 7ff916e26a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:07.988 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:07.986+0000 7ff916e26a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:07.989 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:07.986+0000 7ff916e26a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:30:08.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:30:08.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:30:08.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:30:08.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:30:08.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:30:08.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:30:08.120 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:30:08.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:30:08.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:30:08.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:30:08.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:30:08.348 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:08.346+0000 7ff916e26a40 -1 Falling back to public interface 2026-02-19T13:30:08.984 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:30:08.982+0000 7ff916e26a40 -1 osd.2 0 log_to_monitors true 2026-02-19T13:30:09.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:30:09.276 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:30:09.276 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:30:09.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:30:09.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:30:09.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:30:09.430 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2240782646,v1:127.0.0.1:6819/2240782646] [v2:127.0.0.1:6820/2240782646,v1:127.0.0.1:6821/2240782646] exists,up 565a8be8-b2da-46a8-b2b2-b36b1934ba0a 2026-02-19T13:30:09.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:30:09.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:30:09.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:30:09.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:325: _scrub_abort: create_pool test 1 1 2026-02-19T13:30:09.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-19T13:30:09.622 INFO:tasks.workunit.client.0.trial144.stderr:pool 'test' created 2026-02-19T13:30:09.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:30:10.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:326: _scrub_abort: wait_for_clean 2026-02-19T13:30:10.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:30:10.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:30:10.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:30:10.636 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:30:10.637 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:30:10.637 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:30:10.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:30:10.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:30:10.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:30:10.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:30:10.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:30:10.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:30:10.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:30:10.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:30:10.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:30:10.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:30:10.900 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:30:10.900 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:30:10.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:30:10.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:10.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:30:10.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-19T13:30:10.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-19T13:30:10.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-19T13:30:10.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:10.972 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:30:11.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705669 2026-02-19T13:30:11.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705669 2026-02-19T13:30:11.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-38654705669' 2026-02-19T13:30:11.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:11.049 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:30:11.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542146 2026-02-19T13:30:11.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542146 2026-02-19T13:30:11.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-60129542146' 2026-02-19T13:30:11.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:11.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-19T13:30:11.124 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:11.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:30:11.126 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-19T13:30:11.126 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:11.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-19T13:30:11.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-19T13:30:11.128 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836487 2026-02-19T13:30:11.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:11.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T13:30:11.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:30:12.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:30:12.287 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:12.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836489 -lt 21474836487 2026-02-19T13:30:12.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:12.447 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705669 2026-02-19T13:30:12.447 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:12.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:30:12.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705669 2026-02-19T13:30:12.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:12.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705669 2026-02-19T13:30:12.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-19T13:30:12.451 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705669 2026-02-19T13:30:12.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:30:12.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705670 -lt 38654705669 2026-02-19T13:30:12.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:12.605 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542146 2026-02-19T13:30:12.605 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:12.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:30:12.607 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542146 2026-02-19T13:30:12.607 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:12.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542146 2026-02-19T13:30:12.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-19T13:30:12.609 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542146 2026-02-19T13:30:12.610 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:30:12.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542148 -lt 60129542146 2026-02-19T13:30:12.775 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:30:12.775 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:30:12.775 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:30:12.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:30:12.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:30:12.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:30:12.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:30:12.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:30:12.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:30:12.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:30:12.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:30:13.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:30:13.126 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:30:13.126 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:30:13.126 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:30:13.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:30:13.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:30:13.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:30:13.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:327: _scrub_abort: ceph osd dump 2026-02-19T13:30:13.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:327: _scrub_abort: grep '^pool.*['\'']test['\'']' 2026-02-19T13:30:13.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:327: _scrub_abort: awk '{ print $2 }' 2026-02-19T13:30:13.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:327: _scrub_abort: poolid=1 2026-02-19T13:30:13.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:329: _scrub_abort: dd if=/dev/urandom of=testdata.511299 bs=1032 count=1 2026-02-19T13:30:13.479 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:30:13.479 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:30:13.479 INFO:tasks.workunit.client.0.trial144.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.4674e-05 s, 29.8 MB/s 2026-02-19T13:30:13.479 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: seq 1 1000 2026-02-19T13:30:13.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj1 testdata.511299 2026-02-19T13:30:13.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj2 testdata.511299 2026-02-19T13:30:13.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj3 testdata.511299 2026-02-19T13:30:13.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj4 testdata.511299 2026-02-19T13:30:13.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.572 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj5 testdata.511299 2026-02-19T13:30:13.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj6 testdata.511299 2026-02-19T13:30:13.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj7 testdata.511299 2026-02-19T13:30:13.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj8 testdata.511299 2026-02-19T13:30:13.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj9 testdata.511299 2026-02-19T13:30:13.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj10 testdata.511299 2026-02-19T13:30:13.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj11 testdata.511299 2026-02-19T13:30:13.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj12 testdata.511299 2026-02-19T13:30:13.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj13 testdata.511299 2026-02-19T13:30:13.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj14 testdata.511299 2026-02-19T13:30:13.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj15 testdata.511299 2026-02-19T13:30:13.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj16 testdata.511299 2026-02-19T13:30:13.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj17 testdata.511299 2026-02-19T13:30:13.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj18 testdata.511299 2026-02-19T13:30:13.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj19 testdata.511299 2026-02-19T13:30:13.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj20 testdata.511299 2026-02-19T13:30:13.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj21 testdata.511299 2026-02-19T13:30:13.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:13.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj22 testdata.511299 2026-02-19T13:30:14.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj23 testdata.511299 2026-02-19T13:30:14.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj24 testdata.511299 2026-02-19T13:30:14.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj25 testdata.511299 2026-02-19T13:30:14.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj26 testdata.511299 2026-02-19T13:30:14.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj27 testdata.511299 2026-02-19T13:30:14.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj28 testdata.511299 2026-02-19T13:30:14.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj29 testdata.511299 2026-02-19T13:30:14.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj30 testdata.511299 2026-02-19T13:30:14.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj31 testdata.511299 2026-02-19T13:30:14.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj32 testdata.511299 2026-02-19T13:30:14.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj33 testdata.511299 2026-02-19T13:30:14.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj34 testdata.511299 2026-02-19T13:30:14.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj35 testdata.511299 2026-02-19T13:30:14.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj36 testdata.511299 2026-02-19T13:30:14.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj37 testdata.511299 2026-02-19T13:30:14.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj38 testdata.511299 2026-02-19T13:30:14.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj39 testdata.511299 2026-02-19T13:30:14.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj40 testdata.511299 2026-02-19T13:30:14.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj41 testdata.511299 2026-02-19T13:30:14.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj42 testdata.511299 2026-02-19T13:30:14.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj43 testdata.511299 2026-02-19T13:30:14.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj44 testdata.511299 2026-02-19T13:30:14.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj45 testdata.511299 2026-02-19T13:30:14.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj46 testdata.511299 2026-02-19T13:30:14.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj47 testdata.511299 2026-02-19T13:30:14.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj48 testdata.511299 2026-02-19T13:30:14.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj49 testdata.511299 2026-02-19T13:30:14.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj50 testdata.511299 2026-02-19T13:30:14.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj51 testdata.511299 2026-02-19T13:30:14.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj52 testdata.511299 2026-02-19T13:30:14.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj53 testdata.511299 2026-02-19T13:30:14.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj54 testdata.511299 2026-02-19T13:30:14.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj55 testdata.511299 2026-02-19T13:30:14.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj56 testdata.511299 2026-02-19T13:30:14.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj57 testdata.511299 2026-02-19T13:30:14.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj58 testdata.511299 2026-02-19T13:30:14.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj59 testdata.511299 2026-02-19T13:30:14.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj60 testdata.511299 2026-02-19T13:30:14.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj61 testdata.511299 2026-02-19T13:30:14.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj62 testdata.511299 2026-02-19T13:30:14.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj63 testdata.511299 2026-02-19T13:30:14.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj64 testdata.511299 2026-02-19T13:30:14.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:14.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj65 testdata.511299 2026-02-19T13:30:15.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj66 testdata.511299 2026-02-19T13:30:15.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj67 testdata.511299 2026-02-19T13:30:15.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj68 testdata.511299 2026-02-19T13:30:15.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj69 testdata.511299 2026-02-19T13:30:15.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj70 testdata.511299 2026-02-19T13:30:15.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj71 testdata.511299 2026-02-19T13:30:15.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj72 testdata.511299 2026-02-19T13:30:15.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj73 testdata.511299 2026-02-19T13:30:15.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj74 testdata.511299 2026-02-19T13:30:15.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj75 testdata.511299 2026-02-19T13:30:15.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj76 testdata.511299 2026-02-19T13:30:15.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj77 testdata.511299 2026-02-19T13:30:15.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj78 testdata.511299 2026-02-19T13:30:15.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj79 testdata.511299 2026-02-19T13:30:15.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj80 testdata.511299 2026-02-19T13:30:15.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj81 testdata.511299 2026-02-19T13:30:15.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj82 testdata.511299 2026-02-19T13:30:15.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj83 testdata.511299 2026-02-19T13:30:15.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj84 testdata.511299 2026-02-19T13:30:15.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj85 testdata.511299 2026-02-19T13:30:15.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj86 testdata.511299 2026-02-19T13:30:15.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj87 testdata.511299 2026-02-19T13:30:15.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj88 testdata.511299 2026-02-19T13:30:15.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj89 testdata.511299 2026-02-19T13:30:15.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj90 testdata.511299 2026-02-19T13:30:15.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj91 testdata.511299 2026-02-19T13:30:15.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj92 testdata.511299 2026-02-19T13:30:15.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj93 testdata.511299 2026-02-19T13:30:15.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj94 testdata.511299 2026-02-19T13:30:15.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj95 testdata.511299 2026-02-19T13:30:15.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj96 testdata.511299 2026-02-19T13:30:15.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj97 testdata.511299 2026-02-19T13:30:15.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj98 testdata.511299 2026-02-19T13:30:15.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj99 testdata.511299 2026-02-19T13:30:15.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj100 testdata.511299 2026-02-19T13:30:15.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj101 testdata.511299 2026-02-19T13:30:15.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj102 testdata.511299 2026-02-19T13:30:15.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj103 testdata.511299 2026-02-19T13:30:15.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj104 testdata.511299 2026-02-19T13:30:15.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj105 testdata.511299 2026-02-19T13:30:15.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj106 testdata.511299 2026-02-19T13:30:15.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj107 testdata.511299 2026-02-19T13:30:15.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:15.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj108 testdata.511299 2026-02-19T13:30:16.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj109 testdata.511299 2026-02-19T13:30:16.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.029 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj110 testdata.511299 2026-02-19T13:30:16.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj111 testdata.511299 2026-02-19T13:30:16.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj112 testdata.511299 2026-02-19T13:30:16.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj113 testdata.511299 2026-02-19T13:30:16.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj114 testdata.511299 2026-02-19T13:30:16.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.150 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj115 testdata.511299 2026-02-19T13:30:16.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj116 testdata.511299 2026-02-19T13:30:16.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj117 testdata.511299 2026-02-19T13:30:16.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj118 testdata.511299 2026-02-19T13:30:16.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj119 testdata.511299 2026-02-19T13:30:16.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj120 testdata.511299 2026-02-19T13:30:16.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj121 testdata.511299 2026-02-19T13:30:16.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj122 testdata.511299 2026-02-19T13:30:16.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj123 testdata.511299 2026-02-19T13:30:16.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj124 testdata.511299 2026-02-19T13:30:16.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj125 testdata.511299 2026-02-19T13:30:16.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj126 testdata.511299 2026-02-19T13:30:16.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj127 testdata.511299 2026-02-19T13:30:16.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj128 testdata.511299 2026-02-19T13:30:16.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.490 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj129 testdata.511299 2026-02-19T13:30:16.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj130 testdata.511299 2026-02-19T13:30:16.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.539 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj131 testdata.511299 2026-02-19T13:30:16.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj132 testdata.511299 2026-02-19T13:30:16.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj133 testdata.511299 2026-02-19T13:30:16.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj134 testdata.511299 2026-02-19T13:30:16.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj135 testdata.511299 2026-02-19T13:30:16.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj136 testdata.511299 2026-02-19T13:30:16.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj137 testdata.511299 2026-02-19T13:30:16.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj138 testdata.511299 2026-02-19T13:30:16.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj139 testdata.511299 2026-02-19T13:30:16.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj140 testdata.511299 2026-02-19T13:30:16.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj141 testdata.511299 2026-02-19T13:30:16.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj142 testdata.511299 2026-02-19T13:30:16.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj143 testdata.511299 2026-02-19T13:30:16.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.847 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj144 testdata.511299 2026-02-19T13:30:16.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj145 testdata.511299 2026-02-19T13:30:16.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj146 testdata.511299 2026-02-19T13:30:16.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj147 testdata.511299 2026-02-19T13:30:16.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj148 testdata.511299 2026-02-19T13:30:16.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj149 testdata.511299 2026-02-19T13:30:16.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:16.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj150 testdata.511299 2026-02-19T13:30:17.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj151 testdata.511299 2026-02-19T13:30:17.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj152 testdata.511299 2026-02-19T13:30:17.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj153 testdata.511299 2026-02-19T13:30:17.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj154 testdata.511299 2026-02-19T13:30:17.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj155 testdata.511299 2026-02-19T13:30:17.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj156 testdata.511299 2026-02-19T13:30:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj157 testdata.511299 2026-02-19T13:30:17.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj158 testdata.511299 2026-02-19T13:30:17.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj159 testdata.511299 2026-02-19T13:30:17.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj160 testdata.511299 2026-02-19T13:30:17.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj161 testdata.511299 2026-02-19T13:30:17.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj162 testdata.511299 2026-02-19T13:30:17.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj163 testdata.511299 2026-02-19T13:30:17.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj164 testdata.511299 2026-02-19T13:30:17.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj165 testdata.511299 2026-02-19T13:30:17.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj166 testdata.511299 2026-02-19T13:30:17.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj167 testdata.511299 2026-02-19T13:30:17.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj168 testdata.511299 2026-02-19T13:30:17.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj169 testdata.511299 2026-02-19T13:30:17.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj170 testdata.511299 2026-02-19T13:30:17.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj171 testdata.511299 2026-02-19T13:30:17.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj172 testdata.511299 2026-02-19T13:30:17.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj173 testdata.511299 2026-02-19T13:30:17.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj174 testdata.511299 2026-02-19T13:30:17.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj175 testdata.511299 2026-02-19T13:30:17.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj176 testdata.511299 2026-02-19T13:30:17.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.609 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj177 testdata.511299 2026-02-19T13:30:17.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj178 testdata.511299 2026-02-19T13:30:17.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj179 testdata.511299 2026-02-19T13:30:17.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj180 testdata.511299 2026-02-19T13:30:17.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj181 testdata.511299 2026-02-19T13:30:17.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj182 testdata.511299 2026-02-19T13:30:17.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj183 testdata.511299 2026-02-19T13:30:17.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj184 testdata.511299 2026-02-19T13:30:17.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj185 testdata.511299 2026-02-19T13:30:17.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj186 testdata.511299 2026-02-19T13:30:17.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj187 testdata.511299 2026-02-19T13:30:17.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj188 testdata.511299 2026-02-19T13:30:17.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj189 testdata.511299 2026-02-19T13:30:17.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj190 testdata.511299 2026-02-19T13:30:17.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj191 testdata.511299 2026-02-19T13:30:17.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj192 testdata.511299 2026-02-19T13:30:17.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:17.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj193 testdata.511299 2026-02-19T13:30:18.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj194 testdata.511299 2026-02-19T13:30:18.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj195 testdata.511299 2026-02-19T13:30:18.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj196 testdata.511299 2026-02-19T13:30:18.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj197 testdata.511299 2026-02-19T13:30:18.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj198 testdata.511299 2026-02-19T13:30:18.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj199 testdata.511299 2026-02-19T13:30:18.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj200 testdata.511299 2026-02-19T13:30:18.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj201 testdata.511299 2026-02-19T13:30:18.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj202 testdata.511299 2026-02-19T13:30:18.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj203 testdata.511299 2026-02-19T13:30:18.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj204 testdata.511299 2026-02-19T13:30:18.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj205 testdata.511299 2026-02-19T13:30:18.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj206 testdata.511299 2026-02-19T13:30:18.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj207 testdata.511299 2026-02-19T13:30:18.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj208 testdata.511299 2026-02-19T13:30:18.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj209 testdata.511299 2026-02-19T13:30:18.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj210 testdata.511299 2026-02-19T13:30:18.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj211 testdata.511299 2026-02-19T13:30:18.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj212 testdata.511299 2026-02-19T13:30:18.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj213 testdata.511299 2026-02-19T13:30:18.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj214 testdata.511299 2026-02-19T13:30:18.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj215 testdata.511299 2026-02-19T13:30:18.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj216 testdata.511299 2026-02-19T13:30:18.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj217 testdata.511299 2026-02-19T13:30:18.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj218 testdata.511299 2026-02-19T13:30:18.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj219 testdata.511299 2026-02-19T13:30:18.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.608 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj220 testdata.511299 2026-02-19T13:30:18.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj221 testdata.511299 2026-02-19T13:30:18.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj222 testdata.511299 2026-02-19T13:30:18.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj223 testdata.511299 2026-02-19T13:30:18.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj224 testdata.511299 2026-02-19T13:30:18.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj225 testdata.511299 2026-02-19T13:30:18.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj226 testdata.511299 2026-02-19T13:30:18.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj227 testdata.511299 2026-02-19T13:30:18.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj228 testdata.511299 2026-02-19T13:30:18.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj229 testdata.511299 2026-02-19T13:30:18.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj230 testdata.511299 2026-02-19T13:30:18.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj231 testdata.511299 2026-02-19T13:30:18.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj232 testdata.511299 2026-02-19T13:30:18.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj233 testdata.511299 2026-02-19T13:30:18.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj234 testdata.511299 2026-02-19T13:30:18.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj235 testdata.511299 2026-02-19T13:30:18.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:18.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj236 testdata.511299 2026-02-19T13:30:19.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj237 testdata.511299 2026-02-19T13:30:19.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj238 testdata.511299 2026-02-19T13:30:19.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj239 testdata.511299 2026-02-19T13:30:19.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj240 testdata.511299 2026-02-19T13:30:19.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj241 testdata.511299 2026-02-19T13:30:19.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj242 testdata.511299 2026-02-19T13:30:19.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj243 testdata.511299 2026-02-19T13:30:19.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj244 testdata.511299 2026-02-19T13:30:19.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj245 testdata.511299 2026-02-19T13:30:19.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj246 testdata.511299 2026-02-19T13:30:19.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj247 testdata.511299 2026-02-19T13:30:19.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj248 testdata.511299 2026-02-19T13:30:19.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj249 testdata.511299 2026-02-19T13:30:19.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj250 testdata.511299 2026-02-19T13:30:19.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj251 testdata.511299 2026-02-19T13:30:19.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj252 testdata.511299 2026-02-19T13:30:19.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj253 testdata.511299 2026-02-19T13:30:19.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj254 testdata.511299 2026-02-19T13:30:19.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.435 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj255 testdata.511299 2026-02-19T13:30:19.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj256 testdata.511299 2026-02-19T13:30:19.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj257 testdata.511299 2026-02-19T13:30:19.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj258 testdata.511299 2026-02-19T13:30:19.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj259 testdata.511299 2026-02-19T13:30:19.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj260 testdata.511299 2026-02-19T13:30:19.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj261 testdata.511299 2026-02-19T13:30:19.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj262 testdata.511299 2026-02-19T13:30:19.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj263 testdata.511299 2026-02-19T13:30:19.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj264 testdata.511299 2026-02-19T13:30:19.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj265 testdata.511299 2026-02-19T13:30:19.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj266 testdata.511299 2026-02-19T13:30:19.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj267 testdata.511299 2026-02-19T13:30:19.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj268 testdata.511299 2026-02-19T13:30:19.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj269 testdata.511299 2026-02-19T13:30:19.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj270 testdata.511299 2026-02-19T13:30:19.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj271 testdata.511299 2026-02-19T13:30:19.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj272 testdata.511299 2026-02-19T13:30:19.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj273 testdata.511299 2026-02-19T13:30:19.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj274 testdata.511299 2026-02-19T13:30:19.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj275 testdata.511299 2026-02-19T13:30:19.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj276 testdata.511299 2026-02-19T13:30:19.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj277 testdata.511299 2026-02-19T13:30:19.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj278 testdata.511299 2026-02-19T13:30:19.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj279 testdata.511299 2026-02-19T13:30:19.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:19.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj280 testdata.511299 2026-02-19T13:30:20.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj281 testdata.511299 2026-02-19T13:30:20.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj282 testdata.511299 2026-02-19T13:30:20.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj283 testdata.511299 2026-02-19T13:30:20.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj284 testdata.511299 2026-02-19T13:30:20.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj285 testdata.511299 2026-02-19T13:30:20.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj286 testdata.511299 2026-02-19T13:30:20.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj287 testdata.511299 2026-02-19T13:30:20.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj288 testdata.511299 2026-02-19T13:30:20.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj289 testdata.511299 2026-02-19T13:30:20.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj290 testdata.511299 2026-02-19T13:30:20.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj291 testdata.511299 2026-02-19T13:30:20.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj292 testdata.511299 2026-02-19T13:30:20.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj293 testdata.511299 2026-02-19T13:30:20.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj294 testdata.511299 2026-02-19T13:30:20.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj295 testdata.511299 2026-02-19T13:30:20.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj296 testdata.511299 2026-02-19T13:30:20.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj297 testdata.511299 2026-02-19T13:30:20.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj298 testdata.511299 2026-02-19T13:30:20.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj299 testdata.511299 2026-02-19T13:30:20.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj300 testdata.511299 2026-02-19T13:30:20.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj301 testdata.511299 2026-02-19T13:30:20.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj302 testdata.511299 2026-02-19T13:30:20.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj303 testdata.511299 2026-02-19T13:30:20.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj304 testdata.511299 2026-02-19T13:30:20.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj305 testdata.511299 2026-02-19T13:30:20.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj306 testdata.511299 2026-02-19T13:30:20.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj307 testdata.511299 2026-02-19T13:30:20.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj308 testdata.511299 2026-02-19T13:30:20.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj309 testdata.511299 2026-02-19T13:30:20.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj310 testdata.511299 2026-02-19T13:30:20.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj311 testdata.511299 2026-02-19T13:30:20.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj312 testdata.511299 2026-02-19T13:30:20.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj313 testdata.511299 2026-02-19T13:30:20.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj314 testdata.511299 2026-02-19T13:30:20.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj315 testdata.511299 2026-02-19T13:30:20.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj316 testdata.511299 2026-02-19T13:30:20.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj317 testdata.511299 2026-02-19T13:30:20.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj318 testdata.511299 2026-02-19T13:30:20.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj319 testdata.511299 2026-02-19T13:30:20.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj320 testdata.511299 2026-02-19T13:30:20.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj321 testdata.511299 2026-02-19T13:30:20.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.950 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj322 testdata.511299 2026-02-19T13:30:20.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj323 testdata.511299 2026-02-19T13:30:20.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:20.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj324 testdata.511299 2026-02-19T13:30:21.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj325 testdata.511299 2026-02-19T13:30:21.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj326 testdata.511299 2026-02-19T13:30:21.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj327 testdata.511299 2026-02-19T13:30:21.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj328 testdata.511299 2026-02-19T13:30:21.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj329 testdata.511299 2026-02-19T13:30:21.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj330 testdata.511299 2026-02-19T13:30:21.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj331 testdata.511299 2026-02-19T13:30:21.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj332 testdata.511299 2026-02-19T13:30:21.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj333 testdata.511299 2026-02-19T13:30:21.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj334 testdata.511299 2026-02-19T13:30:21.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj335 testdata.511299 2026-02-19T13:30:21.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj336 testdata.511299 2026-02-19T13:30:21.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj337 testdata.511299 2026-02-19T13:30:21.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj338 testdata.511299 2026-02-19T13:30:21.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj339 testdata.511299 2026-02-19T13:30:21.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj340 testdata.511299 2026-02-19T13:30:21.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj341 testdata.511299 2026-02-19T13:30:21.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj342 testdata.511299 2026-02-19T13:30:21.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj343 testdata.511299 2026-02-19T13:30:21.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj344 testdata.511299 2026-02-19T13:30:21.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj345 testdata.511299 2026-02-19T13:30:21.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj346 testdata.511299 2026-02-19T13:30:21.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj347 testdata.511299 2026-02-19T13:30:21.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj348 testdata.511299 2026-02-19T13:30:21.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj349 testdata.511299 2026-02-19T13:30:21.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj350 testdata.511299 2026-02-19T13:30:21.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj351 testdata.511299 2026-02-19T13:30:21.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj352 testdata.511299 2026-02-19T13:30:21.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj353 testdata.511299 2026-02-19T13:30:21.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj354 testdata.511299 2026-02-19T13:30:21.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj355 testdata.511299 2026-02-19T13:30:21.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj356 testdata.511299 2026-02-19T13:30:21.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj357 testdata.511299 2026-02-19T13:30:21.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj358 testdata.511299 2026-02-19T13:30:21.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj359 testdata.511299 2026-02-19T13:30:21.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj360 testdata.511299 2026-02-19T13:30:21.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj361 testdata.511299 2026-02-19T13:30:21.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj362 testdata.511299 2026-02-19T13:30:21.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj363 testdata.511299 2026-02-19T13:30:21.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj364 testdata.511299 2026-02-19T13:30:21.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj365 testdata.511299 2026-02-19T13:30:21.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj366 testdata.511299 2026-02-19T13:30:21.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:21.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj367 testdata.511299 2026-02-19T13:30:22.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj368 testdata.511299 2026-02-19T13:30:22.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj369 testdata.511299 2026-02-19T13:30:22.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj370 testdata.511299 2026-02-19T13:30:22.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj371 testdata.511299 2026-02-19T13:30:22.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj372 testdata.511299 2026-02-19T13:30:22.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj373 testdata.511299 2026-02-19T13:30:22.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj374 testdata.511299 2026-02-19T13:30:22.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj375 testdata.511299 2026-02-19T13:30:22.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj376 testdata.511299 2026-02-19T13:30:22.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj377 testdata.511299 2026-02-19T13:30:22.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj378 testdata.511299 2026-02-19T13:30:22.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj379 testdata.511299 2026-02-19T13:30:22.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj380 testdata.511299 2026-02-19T13:30:22.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj381 testdata.511299 2026-02-19T13:30:22.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj382 testdata.511299 2026-02-19T13:30:22.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj383 testdata.511299 2026-02-19T13:30:22.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj384 testdata.511299 2026-02-19T13:30:22.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj385 testdata.511299 2026-02-19T13:30:22.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj386 testdata.511299 2026-02-19T13:30:22.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj387 testdata.511299 2026-02-19T13:30:22.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj388 testdata.511299 2026-02-19T13:30:22.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj389 testdata.511299 2026-02-19T13:30:22.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj390 testdata.511299 2026-02-19T13:30:22.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj391 testdata.511299 2026-02-19T13:30:22.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj392 testdata.511299 2026-02-19T13:30:22.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj393 testdata.511299 2026-02-19T13:30:22.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj394 testdata.511299 2026-02-19T13:30:22.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj395 testdata.511299 2026-02-19T13:30:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj396 testdata.511299 2026-02-19T13:30:22.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj397 testdata.511299 2026-02-19T13:30:22.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj398 testdata.511299 2026-02-19T13:30:22.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj399 testdata.511299 2026-02-19T13:30:22.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj400 testdata.511299 2026-02-19T13:30:22.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj401 testdata.511299 2026-02-19T13:30:22.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj402 testdata.511299 2026-02-19T13:30:22.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj403 testdata.511299 2026-02-19T13:30:22.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj404 testdata.511299 2026-02-19T13:30:22.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj405 testdata.511299 2026-02-19T13:30:22.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj406 testdata.511299 2026-02-19T13:30:22.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj407 testdata.511299 2026-02-19T13:30:22.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj408 testdata.511299 2026-02-19T13:30:22.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj409 testdata.511299 2026-02-19T13:30:22.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:22.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj410 testdata.511299 2026-02-19T13:30:23.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj411 testdata.511299 2026-02-19T13:30:23.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj412 testdata.511299 2026-02-19T13:30:23.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj413 testdata.511299 2026-02-19T13:30:23.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj414 testdata.511299 2026-02-19T13:30:23.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj415 testdata.511299 2026-02-19T13:30:23.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj416 testdata.511299 2026-02-19T13:30:23.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj417 testdata.511299 2026-02-19T13:30:23.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj418 testdata.511299 2026-02-19T13:30:23.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj419 testdata.511299 2026-02-19T13:30:23.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj420 testdata.511299 2026-02-19T13:30:23.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj421 testdata.511299 2026-02-19T13:30:23.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj422 testdata.511299 2026-02-19T13:30:23.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj423 testdata.511299 2026-02-19T13:30:23.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj424 testdata.511299 2026-02-19T13:30:23.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj425 testdata.511299 2026-02-19T13:30:23.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj426 testdata.511299 2026-02-19T13:30:23.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj427 testdata.511299 2026-02-19T13:30:23.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj428 testdata.511299 2026-02-19T13:30:23.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj429 testdata.511299 2026-02-19T13:30:23.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj430 testdata.511299 2026-02-19T13:30:23.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj431 testdata.511299 2026-02-19T13:30:23.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj432 testdata.511299 2026-02-19T13:30:23.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.516 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj433 testdata.511299 2026-02-19T13:30:23.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj434 testdata.511299 2026-02-19T13:30:23.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj435 testdata.511299 2026-02-19T13:30:23.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj436 testdata.511299 2026-02-19T13:30:23.613 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj437 testdata.511299 2026-02-19T13:30:23.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj438 testdata.511299 2026-02-19T13:30:23.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj439 testdata.511299 2026-02-19T13:30:23.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj440 testdata.511299 2026-02-19T13:30:23.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj441 testdata.511299 2026-02-19T13:30:23.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj442 testdata.511299 2026-02-19T13:30:23.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj443 testdata.511299 2026-02-19T13:30:23.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj444 testdata.511299 2026-02-19T13:30:23.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj445 testdata.511299 2026-02-19T13:30:23.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj446 testdata.511299 2026-02-19T13:30:23.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj447 testdata.511299 2026-02-19T13:30:23.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj448 testdata.511299 2026-02-19T13:30:23.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj449 testdata.511299 2026-02-19T13:30:23.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj450 testdata.511299 2026-02-19T13:30:23.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj451 testdata.511299 2026-02-19T13:30:23.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj452 testdata.511299 2026-02-19T13:30:23.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj453 testdata.511299 2026-02-19T13:30:23.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:23.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj454 testdata.511299 2026-02-19T13:30:24.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj455 testdata.511299 2026-02-19T13:30:24.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj456 testdata.511299 2026-02-19T13:30:24.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj457 testdata.511299 2026-02-19T13:30:24.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj458 testdata.511299 2026-02-19T13:30:24.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.119 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj459 testdata.511299 2026-02-19T13:30:24.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj460 testdata.511299 2026-02-19T13:30:24.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj461 testdata.511299 2026-02-19T13:30:24.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj462 testdata.511299 2026-02-19T13:30:24.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj463 testdata.511299 2026-02-19T13:30:24.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj464 testdata.511299 2026-02-19T13:30:24.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj465 testdata.511299 2026-02-19T13:30:24.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.284 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj466 testdata.511299 2026-02-19T13:30:24.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj467 testdata.511299 2026-02-19T13:30:24.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj468 testdata.511299 2026-02-19T13:30:24.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj469 testdata.511299 2026-02-19T13:30:24.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj470 testdata.511299 2026-02-19T13:30:24.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj471 testdata.511299 2026-02-19T13:30:24.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj472 testdata.511299 2026-02-19T13:30:24.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj473 testdata.511299 2026-02-19T13:30:24.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj474 testdata.511299 2026-02-19T13:30:24.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj475 testdata.511299 2026-02-19T13:30:24.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj476 testdata.511299 2026-02-19T13:30:24.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj477 testdata.511299 2026-02-19T13:30:24.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj478 testdata.511299 2026-02-19T13:30:24.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj479 testdata.511299 2026-02-19T13:30:24.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj480 testdata.511299 2026-02-19T13:30:24.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj481 testdata.511299 2026-02-19T13:30:24.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj482 testdata.511299 2026-02-19T13:30:24.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj483 testdata.511299 2026-02-19T13:30:24.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj484 testdata.511299 2026-02-19T13:30:24.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj485 testdata.511299 2026-02-19T13:30:24.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj486 testdata.511299 2026-02-19T13:30:24.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj487 testdata.511299 2026-02-19T13:30:24.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj488 testdata.511299 2026-02-19T13:30:24.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj489 testdata.511299 2026-02-19T13:30:24.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj490 testdata.511299 2026-02-19T13:30:24.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj491 testdata.511299 2026-02-19T13:30:24.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj492 testdata.511299 2026-02-19T13:30:24.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj493 testdata.511299 2026-02-19T13:30:24.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj494 testdata.511299 2026-02-19T13:30:24.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj495 testdata.511299 2026-02-19T13:30:24.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:24.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj496 testdata.511299 2026-02-19T13:30:25.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj497 testdata.511299 2026-02-19T13:30:25.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj498 testdata.511299 2026-02-19T13:30:25.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj499 testdata.511299 2026-02-19T13:30:25.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj500 testdata.511299 2026-02-19T13:30:25.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj501 testdata.511299 2026-02-19T13:30:25.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj502 testdata.511299 2026-02-19T13:30:25.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj503 testdata.511299 2026-02-19T13:30:25.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj504 testdata.511299 2026-02-19T13:30:25.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj505 testdata.511299 2026-02-19T13:30:25.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj506 testdata.511299 2026-02-19T13:30:25.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj507 testdata.511299 2026-02-19T13:30:25.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj508 testdata.511299 2026-02-19T13:30:25.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj509 testdata.511299 2026-02-19T13:30:25.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj510 testdata.511299 2026-02-19T13:30:25.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj511 testdata.511299 2026-02-19T13:30:25.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj512 testdata.511299 2026-02-19T13:30:25.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj513 testdata.511299 2026-02-19T13:30:25.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj514 testdata.511299 2026-02-19T13:30:25.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj515 testdata.511299 2026-02-19T13:30:25.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj516 testdata.511299 2026-02-19T13:30:25.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj517 testdata.511299 2026-02-19T13:30:25.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj518 testdata.511299 2026-02-19T13:30:25.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj519 testdata.511299 2026-02-19T13:30:25.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.561 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj520 testdata.511299 2026-02-19T13:30:25.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj521 testdata.511299 2026-02-19T13:30:25.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj522 testdata.511299 2026-02-19T13:30:25.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj523 testdata.511299 2026-02-19T13:30:25.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj524 testdata.511299 2026-02-19T13:30:25.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj525 testdata.511299 2026-02-19T13:30:25.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.692 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj526 testdata.511299 2026-02-19T13:30:25.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj527 testdata.511299 2026-02-19T13:30:25.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj528 testdata.511299 2026-02-19T13:30:25.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj529 testdata.511299 2026-02-19T13:30:25.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj530 testdata.511299 2026-02-19T13:30:25.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj531 testdata.511299 2026-02-19T13:30:25.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj532 testdata.511299 2026-02-19T13:30:25.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj533 testdata.511299 2026-02-19T13:30:25.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj534 testdata.511299 2026-02-19T13:30:25.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj535 testdata.511299 2026-02-19T13:30:25.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj536 testdata.511299 2026-02-19T13:30:25.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj537 testdata.511299 2026-02-19T13:30:25.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.972 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj538 testdata.511299 2026-02-19T13:30:25.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:25.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj539 testdata.511299 2026-02-19T13:30:26.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj540 testdata.511299 2026-02-19T13:30:26.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj541 testdata.511299 2026-02-19T13:30:26.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj542 testdata.511299 2026-02-19T13:30:26.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj543 testdata.511299 2026-02-19T13:30:26.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj544 testdata.511299 2026-02-19T13:30:26.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj545 testdata.511299 2026-02-19T13:30:26.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj546 testdata.511299 2026-02-19T13:30:26.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj547 testdata.511299 2026-02-19T13:30:26.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj548 testdata.511299 2026-02-19T13:30:26.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj549 testdata.511299 2026-02-19T13:30:26.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.251 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj550 testdata.511299 2026-02-19T13:30:26.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj551 testdata.511299 2026-02-19T13:30:26.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj552 testdata.511299 2026-02-19T13:30:26.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj553 testdata.511299 2026-02-19T13:30:26.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj554 testdata.511299 2026-02-19T13:30:26.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.374 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj555 testdata.511299 2026-02-19T13:30:26.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj556 testdata.511299 2026-02-19T13:30:26.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj557 testdata.511299 2026-02-19T13:30:26.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj558 testdata.511299 2026-02-19T13:30:26.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj559 testdata.511299 2026-02-19T13:30:26.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj560 testdata.511299 2026-02-19T13:30:26.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj561 testdata.511299 2026-02-19T13:30:26.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj562 testdata.511299 2026-02-19T13:30:26.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj563 testdata.511299 2026-02-19T13:30:26.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj564 testdata.511299 2026-02-19T13:30:26.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj565 testdata.511299 2026-02-19T13:30:26.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj566 testdata.511299 2026-02-19T13:30:26.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj567 testdata.511299 2026-02-19T13:30:26.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj568 testdata.511299 2026-02-19T13:30:26.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj569 testdata.511299 2026-02-19T13:30:26.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.721 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj570 testdata.511299 2026-02-19T13:30:26.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj571 testdata.511299 2026-02-19T13:30:26.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj572 testdata.511299 2026-02-19T13:30:26.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj573 testdata.511299 2026-02-19T13:30:26.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj574 testdata.511299 2026-02-19T13:30:26.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj575 testdata.511299 2026-02-19T13:30:26.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj576 testdata.511299 2026-02-19T13:30:26.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj577 testdata.511299 2026-02-19T13:30:26.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj578 testdata.511299 2026-02-19T13:30:26.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj579 testdata.511299 2026-02-19T13:30:26.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj580 testdata.511299 2026-02-19T13:30:26.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj581 testdata.511299 2026-02-19T13:30:26.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:26.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj582 testdata.511299 2026-02-19T13:30:27.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj583 testdata.511299 2026-02-19T13:30:27.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.044 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj584 testdata.511299 2026-02-19T13:30:27.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj585 testdata.511299 2026-02-19T13:30:27.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj586 testdata.511299 2026-02-19T13:30:27.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj587 testdata.511299 2026-02-19T13:30:27.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj588 testdata.511299 2026-02-19T13:30:27.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj589 testdata.511299 2026-02-19T13:30:27.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj590 testdata.511299 2026-02-19T13:30:27.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj591 testdata.511299 2026-02-19T13:30:27.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj592 testdata.511299 2026-02-19T13:30:27.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj593 testdata.511299 2026-02-19T13:30:27.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj594 testdata.511299 2026-02-19T13:30:27.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj595 testdata.511299 2026-02-19T13:30:27.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj596 testdata.511299 2026-02-19T13:30:27.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj597 testdata.511299 2026-02-19T13:30:27.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj598 testdata.511299 2026-02-19T13:30:27.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj599 testdata.511299 2026-02-19T13:30:27.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj600 testdata.511299 2026-02-19T13:30:27.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj601 testdata.511299 2026-02-19T13:30:27.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj602 testdata.511299 2026-02-19T13:30:27.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj603 testdata.511299 2026-02-19T13:30:27.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.517 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj604 testdata.511299 2026-02-19T13:30:27.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj605 testdata.511299 2026-02-19T13:30:27.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj606 testdata.511299 2026-02-19T13:30:27.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj607 testdata.511299 2026-02-19T13:30:27.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj608 testdata.511299 2026-02-19T13:30:27.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj609 testdata.511299 2026-02-19T13:30:27.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj610 testdata.511299 2026-02-19T13:30:27.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj611 testdata.511299 2026-02-19T13:30:27.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj612 testdata.511299 2026-02-19T13:30:27.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj613 testdata.511299 2026-02-19T13:30:27.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj614 testdata.511299 2026-02-19T13:30:27.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj615 testdata.511299 2026-02-19T13:30:27.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj616 testdata.511299 2026-02-19T13:30:27.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj617 testdata.511299 2026-02-19T13:30:27.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj618 testdata.511299 2026-02-19T13:30:27.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj619 testdata.511299 2026-02-19T13:30:27.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj620 testdata.511299 2026-02-19T13:30:27.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj621 testdata.511299 2026-02-19T13:30:27.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj622 testdata.511299 2026-02-19T13:30:27.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj623 testdata.511299 2026-02-19T13:30:27.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:27.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj624 testdata.511299 2026-02-19T13:30:28.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj625 testdata.511299 2026-02-19T13:30:28.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj626 testdata.511299 2026-02-19T13:30:28.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj627 testdata.511299 2026-02-19T13:30:28.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj628 testdata.511299 2026-02-19T13:30:28.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj629 testdata.511299 2026-02-19T13:30:28.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj630 testdata.511299 2026-02-19T13:30:28.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj631 testdata.511299 2026-02-19T13:30:28.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj632 testdata.511299 2026-02-19T13:30:28.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj633 testdata.511299 2026-02-19T13:30:28.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj634 testdata.511299 2026-02-19T13:30:28.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj635 testdata.511299 2026-02-19T13:30:28.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj636 testdata.511299 2026-02-19T13:30:28.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj637 testdata.511299 2026-02-19T13:30:28.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj638 testdata.511299 2026-02-19T13:30:28.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj639 testdata.511299 2026-02-19T13:30:28.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj640 testdata.511299 2026-02-19T13:30:28.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj641 testdata.511299 2026-02-19T13:30:28.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj642 testdata.511299 2026-02-19T13:30:28.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj643 testdata.511299 2026-02-19T13:30:28.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj644 testdata.511299 2026-02-19T13:30:28.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj645 testdata.511299 2026-02-19T13:30:28.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj646 testdata.511299 2026-02-19T13:30:28.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj647 testdata.511299 2026-02-19T13:30:28.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj648 testdata.511299 2026-02-19T13:30:28.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.562 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj649 testdata.511299 2026-02-19T13:30:28.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj650 testdata.511299 2026-02-19T13:30:28.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.610 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj651 testdata.511299 2026-02-19T13:30:28.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj652 testdata.511299 2026-02-19T13:30:28.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj653 testdata.511299 2026-02-19T13:30:28.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.677 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj654 testdata.511299 2026-02-19T13:30:28.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj655 testdata.511299 2026-02-19T13:30:28.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj656 testdata.511299 2026-02-19T13:30:28.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj657 testdata.511299 2026-02-19T13:30:28.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj658 testdata.511299 2026-02-19T13:30:28.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj659 testdata.511299 2026-02-19T13:30:28.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj660 testdata.511299 2026-02-19T13:30:28.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj661 testdata.511299 2026-02-19T13:30:28.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.864 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj662 testdata.511299 2026-02-19T13:30:28.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj663 testdata.511299 2026-02-19T13:30:28.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj664 testdata.511299 2026-02-19T13:30:28.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj665 testdata.511299 2026-02-19T13:30:28.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj666 testdata.511299 2026-02-19T13:30:28.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:28.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj667 testdata.511299 2026-02-19T13:30:29.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj668 testdata.511299 2026-02-19T13:30:29.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj669 testdata.511299 2026-02-19T13:30:29.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj670 testdata.511299 2026-02-19T13:30:29.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj671 testdata.511299 2026-02-19T13:30:29.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj672 testdata.511299 2026-02-19T13:30:29.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj673 testdata.511299 2026-02-19T13:30:29.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj674 testdata.511299 2026-02-19T13:30:29.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj675 testdata.511299 2026-02-19T13:30:29.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj676 testdata.511299 2026-02-19T13:30:29.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj677 testdata.511299 2026-02-19T13:30:29.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj678 testdata.511299 2026-02-19T13:30:29.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj679 testdata.511299 2026-02-19T13:30:29.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj680 testdata.511299 2026-02-19T13:30:29.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj681 testdata.511299 2026-02-19T13:30:29.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj682 testdata.511299 2026-02-19T13:30:29.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj683 testdata.511299 2026-02-19T13:30:29.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj684 testdata.511299 2026-02-19T13:30:29.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj685 testdata.511299 2026-02-19T13:30:29.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.400 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj686 testdata.511299 2026-02-19T13:30:29.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.424 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj687 testdata.511299 2026-02-19T13:30:29.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj688 testdata.511299 2026-02-19T13:30:29.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj689 testdata.511299 2026-02-19T13:30:29.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj690 testdata.511299 2026-02-19T13:30:29.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj691 testdata.511299 2026-02-19T13:30:29.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj692 testdata.511299 2026-02-19T13:30:29.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj693 testdata.511299 2026-02-19T13:30:29.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj694 testdata.511299 2026-02-19T13:30:29.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.611 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj695 testdata.511299 2026-02-19T13:30:29.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj696 testdata.511299 2026-02-19T13:30:29.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj697 testdata.511299 2026-02-19T13:30:29.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj698 testdata.511299 2026-02-19T13:30:29.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj699 testdata.511299 2026-02-19T13:30:29.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.731 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj700 testdata.511299 2026-02-19T13:30:29.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj701 testdata.511299 2026-02-19T13:30:29.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj702 testdata.511299 2026-02-19T13:30:29.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj703 testdata.511299 2026-02-19T13:30:29.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj704 testdata.511299 2026-02-19T13:30:29.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj705 testdata.511299 2026-02-19T13:30:29.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj706 testdata.511299 2026-02-19T13:30:29.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj707 testdata.511299 2026-02-19T13:30:29.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj708 testdata.511299 2026-02-19T13:30:29.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.944 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj709 testdata.511299 2026-02-19T13:30:29.967 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj710 testdata.511299 2026-02-19T13:30:29.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:29.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj711 testdata.511299 2026-02-19T13:30:30.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj712 testdata.511299 2026-02-19T13:30:30.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj713 testdata.511299 2026-02-19T13:30:30.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj714 testdata.511299 2026-02-19T13:30:30.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj715 testdata.511299 2026-02-19T13:30:30.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj716 testdata.511299 2026-02-19T13:30:30.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj717 testdata.511299 2026-02-19T13:30:30.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj718 testdata.511299 2026-02-19T13:30:30.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj719 testdata.511299 2026-02-19T13:30:30.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj720 testdata.511299 2026-02-19T13:30:30.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj721 testdata.511299 2026-02-19T13:30:30.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj722 testdata.511299 2026-02-19T13:30:30.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj723 testdata.511299 2026-02-19T13:30:30.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj724 testdata.511299 2026-02-19T13:30:30.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj725 testdata.511299 2026-02-19T13:30:30.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj726 testdata.511299 2026-02-19T13:30:30.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj727 testdata.511299 2026-02-19T13:30:30.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj728 testdata.511299 2026-02-19T13:30:30.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.418 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj729 testdata.511299 2026-02-19T13:30:30.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj730 testdata.511299 2026-02-19T13:30:30.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj731 testdata.511299 2026-02-19T13:30:30.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj732 testdata.511299 2026-02-19T13:30:30.509 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj733 testdata.511299 2026-02-19T13:30:30.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj734 testdata.511299 2026-02-19T13:30:30.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj735 testdata.511299 2026-02-19T13:30:30.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj736 testdata.511299 2026-02-19T13:30:30.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj737 testdata.511299 2026-02-19T13:30:30.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj738 testdata.511299 2026-02-19T13:30:30.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj739 testdata.511299 2026-02-19T13:30:30.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.679 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj740 testdata.511299 2026-02-19T13:30:30.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.704 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj741 testdata.511299 2026-02-19T13:30:30.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj742 testdata.511299 2026-02-19T13:30:30.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj743 testdata.511299 2026-02-19T13:30:30.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj744 testdata.511299 2026-02-19T13:30:30.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj745 testdata.511299 2026-02-19T13:30:30.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.819 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj746 testdata.511299 2026-02-19T13:30:30.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj747 testdata.511299 2026-02-19T13:30:30.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj748 testdata.511299 2026-02-19T13:30:30.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj749 testdata.511299 2026-02-19T13:30:30.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj750 testdata.511299 2026-02-19T13:30:30.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj751 testdata.511299 2026-02-19T13:30:30.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj752 testdata.511299 2026-02-19T13:30:30.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:30.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj753 testdata.511299 2026-02-19T13:30:31.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj754 testdata.511299 2026-02-19T13:30:31.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj755 testdata.511299 2026-02-19T13:30:31.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj756 testdata.511299 2026-02-19T13:30:31.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj757 testdata.511299 2026-02-19T13:30:31.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj758 testdata.511299 2026-02-19T13:30:31.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj759 testdata.511299 2026-02-19T13:30:31.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj760 testdata.511299 2026-02-19T13:30:31.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj761 testdata.511299 2026-02-19T13:30:31.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj762 testdata.511299 2026-02-19T13:30:31.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj763 testdata.511299 2026-02-19T13:30:31.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj764 testdata.511299 2026-02-19T13:30:31.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.291 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj765 testdata.511299 2026-02-19T13:30:31.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj766 testdata.511299 2026-02-19T13:30:31.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj767 testdata.511299 2026-02-19T13:30:31.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj768 testdata.511299 2026-02-19T13:30:31.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj769 testdata.511299 2026-02-19T13:30:31.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj770 testdata.511299 2026-02-19T13:30:31.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj771 testdata.511299 2026-02-19T13:30:31.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj772 testdata.511299 2026-02-19T13:30:31.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj773 testdata.511299 2026-02-19T13:30:31.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj774 testdata.511299 2026-02-19T13:30:31.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj775 testdata.511299 2026-02-19T13:30:31.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj776 testdata.511299 2026-02-19T13:30:31.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.573 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj777 testdata.511299 2026-02-19T13:30:31.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj778 testdata.511299 2026-02-19T13:30:31.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj779 testdata.511299 2026-02-19T13:30:31.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj780 testdata.511299 2026-02-19T13:30:31.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj781 testdata.511299 2026-02-19T13:30:31.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj782 testdata.511299 2026-02-19T13:30:31.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj783 testdata.511299 2026-02-19T13:30:31.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj784 testdata.511299 2026-02-19T13:30:31.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj785 testdata.511299 2026-02-19T13:30:31.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj786 testdata.511299 2026-02-19T13:30:31.808 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj787 testdata.511299 2026-02-19T13:30:31.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj788 testdata.511299 2026-02-19T13:30:31.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj789 testdata.511299 2026-02-19T13:30:31.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj790 testdata.511299 2026-02-19T13:30:31.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj791 testdata.511299 2026-02-19T13:30:31.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj792 testdata.511299 2026-02-19T13:30:31.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj793 testdata.511299 2026-02-19T13:30:31.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:31.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj794 testdata.511299 2026-02-19T13:30:32.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj795 testdata.511299 2026-02-19T13:30:32.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj796 testdata.511299 2026-02-19T13:30:32.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj797 testdata.511299 2026-02-19T13:30:32.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj798 testdata.511299 2026-02-19T13:30:32.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj799 testdata.511299 2026-02-19T13:30:32.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj800 testdata.511299 2026-02-19T13:30:32.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj801 testdata.511299 2026-02-19T13:30:32.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj802 testdata.511299 2026-02-19T13:30:32.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj803 testdata.511299 2026-02-19T13:30:32.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj804 testdata.511299 2026-02-19T13:30:32.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj805 testdata.511299 2026-02-19T13:30:32.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj806 testdata.511299 2026-02-19T13:30:32.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj807 testdata.511299 2026-02-19T13:30:32.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj808 testdata.511299 2026-02-19T13:30:32.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj809 testdata.511299 2026-02-19T13:30:32.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj810 testdata.511299 2026-02-19T13:30:32.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj811 testdata.511299 2026-02-19T13:30:32.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj812 testdata.511299 2026-02-19T13:30:32.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj813 testdata.511299 2026-02-19T13:30:32.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj814 testdata.511299 2026-02-19T13:30:32.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj815 testdata.511299 2026-02-19T13:30:32.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj816 testdata.511299 2026-02-19T13:30:32.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj817 testdata.511299 2026-02-19T13:30:32.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj818 testdata.511299 2026-02-19T13:30:32.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj819 testdata.511299 2026-02-19T13:30:32.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj820 testdata.511299 2026-02-19T13:30:32.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj821 testdata.511299 2026-02-19T13:30:32.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj822 testdata.511299 2026-02-19T13:30:32.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj823 testdata.511299 2026-02-19T13:30:32.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj824 testdata.511299 2026-02-19T13:30:32.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj825 testdata.511299 2026-02-19T13:30:32.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj826 testdata.511299 2026-02-19T13:30:32.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj827 testdata.511299 2026-02-19T13:30:32.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.780 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj828 testdata.511299 2026-02-19T13:30:32.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.805 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj829 testdata.511299 2026-02-19T13:30:32.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj830 testdata.511299 2026-02-19T13:30:32.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj831 testdata.511299 2026-02-19T13:30:32.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj832 testdata.511299 2026-02-19T13:30:32.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj833 testdata.511299 2026-02-19T13:30:32.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj834 testdata.511299 2026-02-19T13:30:32.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj835 testdata.511299 2026-02-19T13:30:32.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj836 testdata.511299 2026-02-19T13:30:32.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:32.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj837 testdata.511299 2026-02-19T13:30:33.005 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.006 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj838 testdata.511299 2026-02-19T13:30:33.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj839 testdata.511299 2026-02-19T13:30:33.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj840 testdata.511299 2026-02-19T13:30:33.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj841 testdata.511299 2026-02-19T13:30:33.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.103 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj842 testdata.511299 2026-02-19T13:30:33.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj843 testdata.511299 2026-02-19T13:30:33.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj844 testdata.511299 2026-02-19T13:30:33.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj845 testdata.511299 2026-02-19T13:30:33.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj846 testdata.511299 2026-02-19T13:30:33.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj847 testdata.511299 2026-02-19T13:30:33.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj848 testdata.511299 2026-02-19T13:30:33.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj849 testdata.511299 2026-02-19T13:30:33.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj850 testdata.511299 2026-02-19T13:30:33.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj851 testdata.511299 2026-02-19T13:30:33.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj852 testdata.511299 2026-02-19T13:30:33.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj853 testdata.511299 2026-02-19T13:30:33.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj854 testdata.511299 2026-02-19T13:30:33.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj855 testdata.511299 2026-02-19T13:30:33.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj856 testdata.511299 2026-02-19T13:30:33.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj857 testdata.511299 2026-02-19T13:30:33.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj858 testdata.511299 2026-02-19T13:30:33.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj859 testdata.511299 2026-02-19T13:30:33.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj860 testdata.511299 2026-02-19T13:30:33.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj861 testdata.511299 2026-02-19T13:30:33.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj862 testdata.511299 2026-02-19T13:30:33.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj863 testdata.511299 2026-02-19T13:30:33.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj864 testdata.511299 2026-02-19T13:30:33.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj865 testdata.511299 2026-02-19T13:30:33.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj866 testdata.511299 2026-02-19T13:30:33.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj867 testdata.511299 2026-02-19T13:30:33.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj868 testdata.511299 2026-02-19T13:30:33.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj869 testdata.511299 2026-02-19T13:30:33.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj870 testdata.511299 2026-02-19T13:30:33.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj871 testdata.511299 2026-02-19T13:30:33.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj872 testdata.511299 2026-02-19T13:30:33.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj873 testdata.511299 2026-02-19T13:30:33.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj874 testdata.511299 2026-02-19T13:30:33.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj875 testdata.511299 2026-02-19T13:30:33.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.923 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj876 testdata.511299 2026-02-19T13:30:33.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj877 testdata.511299 2026-02-19T13:30:33.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj878 testdata.511299 2026-02-19T13:30:33.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:33.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj879 testdata.511299 2026-02-19T13:30:34.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj880 testdata.511299 2026-02-19T13:30:34.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj881 testdata.511299 2026-02-19T13:30:34.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj882 testdata.511299 2026-02-19T13:30:34.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj883 testdata.511299 2026-02-19T13:30:34.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj884 testdata.511299 2026-02-19T13:30:34.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj885 testdata.511299 2026-02-19T13:30:34.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj886 testdata.511299 2026-02-19T13:30:34.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj887 testdata.511299 2026-02-19T13:30:34.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj888 testdata.511299 2026-02-19T13:30:34.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj889 testdata.511299 2026-02-19T13:30:34.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj890 testdata.511299 2026-02-19T13:30:34.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj891 testdata.511299 2026-02-19T13:30:34.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj892 testdata.511299 2026-02-19T13:30:34.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj893 testdata.511299 2026-02-19T13:30:34.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj894 testdata.511299 2026-02-19T13:30:34.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj895 testdata.511299 2026-02-19T13:30:34.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj896 testdata.511299 2026-02-19T13:30:34.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj897 testdata.511299 2026-02-19T13:30:34.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj898 testdata.511299 2026-02-19T13:30:34.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj899 testdata.511299 2026-02-19T13:30:34.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj900 testdata.511299 2026-02-19T13:30:34.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj901 testdata.511299 2026-02-19T13:30:34.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj902 testdata.511299 2026-02-19T13:30:34.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj903 testdata.511299 2026-02-19T13:30:34.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj904 testdata.511299 2026-02-19T13:30:34.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj905 testdata.511299 2026-02-19T13:30:34.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj906 testdata.511299 2026-02-19T13:30:34.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj907 testdata.511299 2026-02-19T13:30:34.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.678 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj908 testdata.511299 2026-02-19T13:30:34.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj909 testdata.511299 2026-02-19T13:30:34.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj910 testdata.511299 2026-02-19T13:30:34.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.751 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj911 testdata.511299 2026-02-19T13:30:34.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj912 testdata.511299 2026-02-19T13:30:34.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj913 testdata.511299 2026-02-19T13:30:34.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj914 testdata.511299 2026-02-19T13:30:34.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj915 testdata.511299 2026-02-19T13:30:34.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj916 testdata.511299 2026-02-19T13:30:34.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj917 testdata.511299 2026-02-19T13:30:34.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj918 testdata.511299 2026-02-19T13:30:34.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj919 testdata.511299 2026-02-19T13:30:34.952 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj920 testdata.511299 2026-02-19T13:30:34.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:34.978 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj921 testdata.511299 2026-02-19T13:30:35.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj922 testdata.511299 2026-02-19T13:30:35.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.028 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj923 testdata.511299 2026-02-19T13:30:35.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj924 testdata.511299 2026-02-19T13:30:35.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.077 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj925 testdata.511299 2026-02-19T13:30:35.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj926 testdata.511299 2026-02-19T13:30:35.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj927 testdata.511299 2026-02-19T13:30:35.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj928 testdata.511299 2026-02-19T13:30:35.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj929 testdata.511299 2026-02-19T13:30:35.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj930 testdata.511299 2026-02-19T13:30:35.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj931 testdata.511299 2026-02-19T13:30:35.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj932 testdata.511299 2026-02-19T13:30:35.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj933 testdata.511299 2026-02-19T13:30:35.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj934 testdata.511299 2026-02-19T13:30:35.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj935 testdata.511299 2026-02-19T13:30:35.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj936 testdata.511299 2026-02-19T13:30:35.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj937 testdata.511299 2026-02-19T13:30:35.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj938 testdata.511299 2026-02-19T13:30:35.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj939 testdata.511299 2026-02-19T13:30:35.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj940 testdata.511299 2026-02-19T13:30:35.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj941 testdata.511299 2026-02-19T13:30:35.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj942 testdata.511299 2026-02-19T13:30:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj943 testdata.511299 2026-02-19T13:30:35.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj944 testdata.511299 2026-02-19T13:30:35.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj945 testdata.511299 2026-02-19T13:30:35.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj946 testdata.511299 2026-02-19T13:30:35.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj947 testdata.511299 2026-02-19T13:30:35.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj948 testdata.511299 2026-02-19T13:30:35.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj949 testdata.511299 2026-02-19T13:30:35.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj950 testdata.511299 2026-02-19T13:30:35.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj951 testdata.511299 2026-02-19T13:30:35.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj952 testdata.511299 2026-02-19T13:30:35.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj953 testdata.511299 2026-02-19T13:30:35.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj954 testdata.511299 2026-02-19T13:30:35.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.787 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj955 testdata.511299 2026-02-19T13:30:35.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj956 testdata.511299 2026-02-19T13:30:35.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj957 testdata.511299 2026-02-19T13:30:35.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj958 testdata.511299 2026-02-19T13:30:35.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj959 testdata.511299 2026-02-19T13:30:35.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.910 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj960 testdata.511299 2026-02-19T13:30:35.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj961 testdata.511299 2026-02-19T13:30:35.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj962 testdata.511299 2026-02-19T13:30:35.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:35.983 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj963 testdata.511299 2026-02-19T13:30:36.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj964 testdata.511299 2026-02-19T13:30:36.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj965 testdata.511299 2026-02-19T13:30:36.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj966 testdata.511299 2026-02-19T13:30:36.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj967 testdata.511299 2026-02-19T13:30:36.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj968 testdata.511299 2026-02-19T13:30:36.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj969 testdata.511299 2026-02-19T13:30:36.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj970 testdata.511299 2026-02-19T13:30:36.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj971 testdata.511299 2026-02-19T13:30:36.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj972 testdata.511299 2026-02-19T13:30:36.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj973 testdata.511299 2026-02-19T13:30:36.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj974 testdata.511299 2026-02-19T13:30:36.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj975 testdata.511299 2026-02-19T13:30:36.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj976 testdata.511299 2026-02-19T13:30:36.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj977 testdata.511299 2026-02-19T13:30:36.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj978 testdata.511299 2026-02-19T13:30:36.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj979 testdata.511299 2026-02-19T13:30:36.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj980 testdata.511299 2026-02-19T13:30:36.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj981 testdata.511299 2026-02-19T13:30:36.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj982 testdata.511299 2026-02-19T13:30:36.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj983 testdata.511299 2026-02-19T13:30:36.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj984 testdata.511299 2026-02-19T13:30:36.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj985 testdata.511299 2026-02-19T13:30:36.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj986 testdata.511299 2026-02-19T13:30:36.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj987 testdata.511299 2026-02-19T13:30:36.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.570 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj988 testdata.511299 2026-02-19T13:30:36.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj989 testdata.511299 2026-02-19T13:30:36.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj990 testdata.511299 2026-02-19T13:30:36.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj991 testdata.511299 2026-02-19T13:30:36.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj992 testdata.511299 2026-02-19T13:30:36.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj993 testdata.511299 2026-02-19T13:30:36.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.706 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj994 testdata.511299 2026-02-19T13:30:36.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj995 testdata.511299 2026-02-19T13:30:36.754 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj996 testdata.511299 2026-02-19T13:30:36.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.779 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj997 testdata.511299 2026-02-19T13:30:36.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj998 testdata.511299 2026-02-19T13:30:36.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj999 testdata.511299 2026-02-19T13:30:36.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: for i in `seq 1 $objects` 2026-02-19T13:30:36.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: rados -p test put obj1000 testdata.511299 2026-02-19T13:30:36.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:334: _scrub_abort: rm -f testdata.511299 2026-02-19T13:30:36.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:336: _scrub_abort: get_primary test obj1 2026-02-19T13:30:36.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=test 2026-02-19T13:30:36.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T13:30:36.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map test obj1 2026-02-19T13:30:36.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:30:37.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:336: _scrub_abort: local primary=1 2026-02-19T13:30:37.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: local pgid=1.0 2026-02-19T13:30:37.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:339: _scrub_abort: ceph tell 1.0 schedule-scrub 2026-02-19T13:30:37.099 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:30:37.099 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:30:37.099 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:30:37.100 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-17T13:30:37.100168+0000" 2026-02-19T13:30:37.100 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:30:37.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:342: _scrub_abort: set -o pipefail 2026-02-19T13:30:37.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:343: _scrub_abort: found=no 2026-02-19T13:30:37.107 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:344: _scrub_abort: seq 0 200 2026-02-19T13:30:37.107 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:344: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:30:37.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:346: _scrub_abort: flush_pg_stats 2026-02-19T13:30:37.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:30:37.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:30:37.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:30:37.264 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:30:37.264 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:30:37.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:30:37.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:37.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:30:37.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836515 2026-02-19T13:30:37.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836515 2026-02-19T13:30:37.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836515' 2026-02-19T13:30:37.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:37.337 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:30:37.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705696 2026-02-19T13:30:37.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705696 2026-02-19T13:30:37.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836515 1-38654705696' 2026-02-19T13:30:37.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:37.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:30:37.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542174 2026-02-19T13:30:37.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542174 2026-02-19T13:30:37.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836515 1-38654705696 2-60129542174' 2026-02-19T13:30:37.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:37.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836515 2026-02-19T13:30:37.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:37.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:30:37.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836515 2026-02-19T13:30:37.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:37.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836515 2026-02-19T13:30:37.486 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836515 2026-02-19T13:30:37.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836515' 2026-02-19T13:30:37.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:37.650 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836513 -lt 21474836515 2026-02-19T13:30:37.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:30:38.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:30:38.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:38.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836516 -lt 21474836515 2026-02-19T13:30:38.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:38.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705696 2026-02-19T13:30:38.814 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:38.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:30:38.816 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705696 2026-02-19T13:30:38.816 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:38.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705696 2026-02-19T13:30:38.818 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705696' 2026-02-19T13:30:38.818 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705696 2026-02-19T13:30:38.819 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:30:38.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705697 -lt 38654705696 2026-02-19T13:30:38.986 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:38.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542174 2026-02-19T13:30:38.986 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:38.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:30:38.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542174 2026-02-19T13:30:38.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:38.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542174 2026-02-19T13:30:38.990 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542174 2026-02-19T13:30:38.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542174' 2026-02-19T13:30:38.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:30:39.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542175 -lt 60129542174 2026-02-19T13:30:39.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: ceph pg dump pgs 2026-02-19T13:30:39.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: grep '^1.0' 2026-02-19T13:30:39.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: grep -q scrubbing 2026-02-19T13:30:39.305 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:30:39.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:344: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:30:39.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:346: _scrub_abort: flush_pg_stats 2026-02-19T13:30:39.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:30:39.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:30:39.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:30:39.469 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:30:39.470 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:30:39.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:30:39.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:39.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:30:39.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836518 2026-02-19T13:30:39.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836518 2026-02-19T13:30:39.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518' 2026-02-19T13:30:39.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:39.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:30:39.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705700 2026-02-19T13:30:39.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705700 2026-02-19T13:30:39.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518 1-38654705700' 2026-02-19T13:30:39.612 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:39.613 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:30:39.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542177 2026-02-19T13:30:39.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542177 2026-02-19T13:30:39.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836518 1-38654705700 2-60129542177' 2026-02-19T13:30:39.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:39.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836518 2026-02-19T13:30:39.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:39.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:30:39.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836518 2026-02-19T13:30:39.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:39.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836518 2026-02-19T13:30:39.689 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836518 2026-02-19T13:30:39.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836518' 2026-02-19T13:30:39.689 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:39.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836516 -lt 21474836518 2026-02-19T13:30:39.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:30:40.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:30:40.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:41.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836519 -lt 21474836518 2026-02-19T13:30:41.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:41.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705700 2026-02-19T13:30:41.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:41.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:30:41.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:41.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705700 2026-02-19T13:30:41.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705700 2026-02-19T13:30:41.019 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705700 2026-02-19T13:30:41.019 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705700' 2026-02-19T13:30:41.020 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:30:41.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705700 -lt 38654705700 2026-02-19T13:30:41.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:41.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542177 2026-02-19T13:30:41.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:41.177 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:30:41.177 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542177 2026-02-19T13:30:41.178 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:41.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542177 2026-02-19T13:30:41.179 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542177 2026-02-19T13:30:41.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542177' 2026-02-19T13:30:41.180 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:30:41.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542178 -lt 60129542177 2026-02-19T13:30:41.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: ceph pg dump pgs 2026-02-19T13:30:41.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: grep '^1.0' 2026-02-19T13:30:41.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: grep -q scrubbing 2026-02-19T13:30:41.488 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:30:41.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:344: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:30:41.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:346: _scrub_abort: flush_pg_stats 2026-02-19T13:30:41.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:30:41.498 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:30:41.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:30:41.653 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:30:41.653 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:30:41.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:30:41.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:41.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:30:41.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836521 2026-02-19T13:30:41.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836521 2026-02-19T13:30:41.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521' 2026-02-19T13:30:41.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:41.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:30:41.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705703 2026-02-19T13:30:41.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705703 2026-02-19T13:30:41.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-38654705703' 2026-02-19T13:30:41.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:41.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:30:41.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542180 2026-02-19T13:30:41.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542180 2026-02-19T13:30:41.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-38654705703 2-60129542180' 2026-02-19T13:30:41.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:41.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836521 2026-02-19T13:30:41.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:41.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:30:41.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836521 2026-02-19T13:30:41.874 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:41.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836521 2026-02-19T13:30:41.875 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836521 2026-02-19T13:30:41.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836521' 2026-02-19T13:30:41.876 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:42.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836519 -lt 21474836521 2026-02-19T13:30:42.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:30:43.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:30:43.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:43.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836522 -lt 21474836521 2026-02-19T13:30:43.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:43.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705703 2026-02-19T13:30:43.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:43.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:30:43.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705703 2026-02-19T13:30:43.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:43.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705703 2026-02-19T13:30:43.194 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705703 2026-02-19T13:30:43.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705703' 2026-02-19T13:30:43.194 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:30:43.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705703 -lt 38654705703 2026-02-19T13:30:43.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:43.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542180 2026-02-19T13:30:43.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:43.362 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:30:43.363 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542180 2026-02-19T13:30:43.363 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:43.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542180 2026-02-19T13:30:43.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542180' 2026-02-19T13:30:43.365 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542180 2026-02-19T13:30:43.365 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:30:43.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542181 -lt 60129542180 2026-02-19T13:30:43.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: ceph pg dump pgs 2026-02-19T13:30:43.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: grep '^1.0' 2026-02-19T13:30:43.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: grep -q scrubbing 2026-02-19T13:30:43.667 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:30:43.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:349: _scrub_abort: found=yes 2026-02-19T13:30:43.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:351: _scrub_abort: break 2026-02-19T13:30:43.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:354: _scrub_abort: set +o pipefail 2026-02-19T13:30:43.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:356: _scrub_abort: test yes = no 2026-02-19T13:30:43.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:362: _scrub_abort: ceph osd set noscrub 2026-02-19T13:30:43.874 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is set 2026-02-19T13:30:43.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:363: _scrub_abort: '[' scrub = deep-scrub ']' 2026-02-19T13:30:43.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:369: _scrub_abort: set -o pipefail 2026-02-19T13:30:43.890 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: seq 0 200 2026-02-19T13:30:43.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:30:43.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:30:43.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:30:43.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:30:44.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:30:44.047 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:30:44.047 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:30:44.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:30:44.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:44.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:30:44.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836525 2026-02-19T13:30:44.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836525 2026-02-19T13:30:44.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836525' 2026-02-19T13:30:44.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:44.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:30:44.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705706 2026-02-19T13:30:44.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705706 2026-02-19T13:30:44.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836525 1-38654705706' 2026-02-19T13:30:44.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:44.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:30:44.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542184 2026-02-19T13:30:44.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542184 2026-02-19T13:30:44.258 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836525 1-38654705706 2-60129542184' 2026-02-19T13:30:44.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:44.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836525 2026-02-19T13:30:44.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:44.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:30:44.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836525 2026-02-19T13:30:44.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:44.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836525 2026-02-19T13:30:44.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836525' 2026-02-19T13:30:44.263 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836525 2026-02-19T13:30:44.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:44.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836525 -lt 21474836525 2026-02-19T13:30:44.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:44.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705706 2026-02-19T13:30:44.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:44.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:30:44.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705706 2026-02-19T13:30:44.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:44.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705706 2026-02-19T13:30:44.432 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705706 2026-02-19T13:30:44.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705706' 2026-02-19T13:30:44.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:30:44.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705706 -lt 38654705706 2026-02-19T13:30:44.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:44.601 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542184 2026-02-19T13:30:44.601 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:44.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:30:44.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542184 2026-02-19T13:30:44.603 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:44.604 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542184 2026-02-19T13:30:44.604 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542184 2026-02-19T13:30:44.605 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542184' 2026-02-19T13:30:44.605 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:30:44.771 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542184 -lt 60129542184 2026-02-19T13:30:44.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:30:44.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:30:44.772 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:30:44.919 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:30:44.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:30:44.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:30:44.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:30:44.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:30:44.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:30:45.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:30:45.087 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:30:45.087 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:30:45.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:30:45.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:45.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:30:45.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836527 2026-02-19T13:30:45.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836527 2026-02-19T13:30:45.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836527' 2026-02-19T13:30:45.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:45.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:30:45.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705708 2026-02-19T13:30:45.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705708 2026-02-19T13:30:45.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836527 1-38654705708' 2026-02-19T13:30:45.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:45.231 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:30:45.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542186 2026-02-19T13:30:45.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542186 2026-02-19T13:30:45.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836527 1-38654705708 2-60129542186' 2026-02-19T13:30:45.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:45.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836527 2026-02-19T13:30:45.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:45.302 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:30:45.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836527 2026-02-19T13:30:45.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:45.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836527 2026-02-19T13:30:45.304 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836527 2026-02-19T13:30:45.305 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836527' 2026-02-19T13:30:45.305 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:45.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836525 -lt 21474836527 2026-02-19T13:30:45.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:30:46.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:30:46.475 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:46.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836528 -lt 21474836527 2026-02-19T13:30:46.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:46.636 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705708 2026-02-19T13:30:46.636 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:46.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:30:46.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705708 2026-02-19T13:30:46.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:46.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705708 2026-02-19T13:30:46.639 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705708' 2026-02-19T13:30:46.640 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705708 2026-02-19T13:30:46.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:30:46.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705709 -lt 38654705708 2026-02-19T13:30:46.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:46.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542186 2026-02-19T13:30:46.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:46.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:30:46.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542186 2026-02-19T13:30:46.800 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:46.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542186 2026-02-19T13:30:46.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542186' 2026-02-19T13:30:46.802 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542186 2026-02-19T13:30:46.802 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:30:46.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542187 -lt 60129542186 2026-02-19T13:30:46.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:30:46.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:30:46.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:30:47.116 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:30:47.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:30:47.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:30:47.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:30:47.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:30:47.126 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:30:47.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:30:47.282 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:30:47.282 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:30:47.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:30:47.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:47.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:30:47.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836530 2026-02-19T13:30:47.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836530 2026-02-19T13:30:47.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836530' 2026-02-19T13:30:47.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:47.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:30:47.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705711 2026-02-19T13:30:47.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705711 2026-02-19T13:30:47.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836530 1-38654705711' 2026-02-19T13:30:47.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:47.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:30:47.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542189 2026-02-19T13:30:47.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542189 2026-02-19T13:30:47.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836530 1-38654705711 2-60129542189' 2026-02-19T13:30:47.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:47.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836530 2026-02-19T13:30:47.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:47.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:30:47.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836530 2026-02-19T13:30:47.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:47.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836530 2026-02-19T13:30:47.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836530' 2026-02-19T13:30:47.507 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836530 2026-02-19T13:30:47.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:47.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836528 -lt 21474836530 2026-02-19T13:30:47.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:30:48.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:30:48.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:48.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836531 -lt 21474836530 2026-02-19T13:30:48.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:48.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705711 2026-02-19T13:30:48.832 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:48.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:30:48.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705711 2026-02-19T13:30:48.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:48.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705711 2026-02-19T13:30:48.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705711' 2026-02-19T13:30:48.836 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705711 2026-02-19T13:30:48.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:30:48.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705712 -lt 38654705711 2026-02-19T13:30:48.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:48.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542189 2026-02-19T13:30:48.998 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:48.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:30:48.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542189 2026-02-19T13:30:49.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:49.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542189 2026-02-19T13:30:49.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542189' 2026-02-19T13:30:49.001 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542189 2026-02-19T13:30:49.002 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:30:49.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542190 -lt 60129542189 2026-02-19T13:30:49.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:30:49.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:30:49.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:30:49.317 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:30:49.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:30:49.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:30:49.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:30:49.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:30:49.328 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:30:49.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:30:49.488 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:30:49.488 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:30:49.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:30:49.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:49.488 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:30:49.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836533 2026-02-19T13:30:49.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836533 2026-02-19T13:30:49.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836533' 2026-02-19T13:30:49.564 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:49.564 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:30:49.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705715 2026-02-19T13:30:49.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705715 2026-02-19T13:30:49.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836533 1-38654705715' 2026-02-19T13:30:49.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:49.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:30:49.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542192 2026-02-19T13:30:49.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542192 2026-02-19T13:30:49.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836533 1-38654705715 2-60129542192' 2026-02-19T13:30:49.705 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:49.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836533 2026-02-19T13:30:49.706 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:49.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:30:49.707 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836533 2026-02-19T13:30:49.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:49.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836533 2026-02-19T13:30:49.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836533' 2026-02-19T13:30:49.709 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836533 2026-02-19T13:30:49.710 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:49.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836531 -lt 21474836533 2026-02-19T13:30:49.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:30:50.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:30:50.877 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:51.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836534 -lt 21474836533 2026-02-19T13:30:51.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:51.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705715 2026-02-19T13:30:51.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:51.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:30:51.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705715 2026-02-19T13:30:51.039 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:51.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705715 2026-02-19T13:30:51.040 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705715 2026-02-19T13:30:51.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705715' 2026-02-19T13:30:51.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:30:51.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705715 -lt 38654705715 2026-02-19T13:30:51.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:51.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542192 2026-02-19T13:30:51.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:51.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:30:51.210 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542192 2026-02-19T13:30:51.210 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:51.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542192 2026-02-19T13:30:51.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542192' 2026-02-19T13:30:51.212 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542192 2026-02-19T13:30:51.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:30:51.367 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542193 -lt 60129542192 2026-02-19T13:30:51.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:30:51.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:30:51.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:30:51.514 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:30:51.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:30:51.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:30:51.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:30:51.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:30:51.525 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:30:51.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:30:51.684 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:30:51.684 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:30:51.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:30:51.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:51.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:30:51.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836536 2026-02-19T13:30:51.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836536 2026-02-19T13:30:51.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836536' 2026-02-19T13:30:51.757 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:51.757 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:30:51.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705718 2026-02-19T13:30:51.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705718 2026-02-19T13:30:51.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836536 1-38654705718' 2026-02-19T13:30:51.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:51.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:30:51.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542195 2026-02-19T13:30:51.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542195 2026-02-19T13:30:51.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836536 1-38654705718 2-60129542195' 2026-02-19T13:30:51.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:51.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836536 2026-02-19T13:30:51.903 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:51.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:30:51.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836536 2026-02-19T13:30:51.904 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:51.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836536 2026-02-19T13:30:51.906 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836536 2026-02-19T13:30:51.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836536' 2026-02-19T13:30:51.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:52.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836534 -lt 21474836536 2026-02-19T13:30:52.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:30:53.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:30:53.080 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:53.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836537 -lt 21474836536 2026-02-19T13:30:53.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:53.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705718 2026-02-19T13:30:53.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:53.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:30:53.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705718 2026-02-19T13:30:53.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:53.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705718 2026-02-19T13:30:53.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705718' 2026-02-19T13:30:53.243 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705718 2026-02-19T13:30:53.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:30:53.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705718 -lt 38654705718 2026-02-19T13:30:53.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:53.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542195 2026-02-19T13:30:53.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:53.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:30:53.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542195 2026-02-19T13:30:53.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:53.407 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542195 2026-02-19T13:30:53.407 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542195 2026-02-19T13:30:53.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542195' 2026-02-19T13:30:53.408 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:30:53.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542196 -lt 60129542195 2026-02-19T13:30:53.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:30:53.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:30:53.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:30:53.712 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:30:53.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:30:53.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:30:53.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:30:53.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:30:53.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:30:53.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:30:53.878 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:30:53.878 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:30:53.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:30:53.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:53.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:30:53.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836539 2026-02-19T13:30:53.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836539 2026-02-19T13:30:53.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836539' 2026-02-19T13:30:53.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:53.949 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:30:54.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705721 2026-02-19T13:30:54.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705721 2026-02-19T13:30:54.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836539 1-38654705721' 2026-02-19T13:30:54.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:54.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:30:54.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542198 2026-02-19T13:30:54.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542198 2026-02-19T13:30:54.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836539 1-38654705721 2-60129542198' 2026-02-19T13:30:54.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:54.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836539 2026-02-19T13:30:54.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:54.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:30:54.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836539 2026-02-19T13:30:54.096 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:54.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836539 2026-02-19T13:30:54.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836539' 2026-02-19T13:30:54.098 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836539 2026-02-19T13:30:54.098 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:54.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836537 -lt 21474836539 2026-02-19T13:30:54.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:30:55.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:30:55.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:55.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836540 -lt 21474836539 2026-02-19T13:30:55.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:55.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705721 2026-02-19T13:30:55.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:55.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:30:55.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705721 2026-02-19T13:30:55.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:55.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705721 2026-02-19T13:30:55.428 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705721 2026-02-19T13:30:55.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705721' 2026-02-19T13:30:55.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:30:55.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705721 -lt 38654705721 2026-02-19T13:30:55.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:55.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542198 2026-02-19T13:30:55.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:55.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:30:55.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542198 2026-02-19T13:30:55.597 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:55.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542198 2026-02-19T13:30:55.598 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542198 2026-02-19T13:30:55.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542198' 2026-02-19T13:30:55.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:30:55.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542199 -lt 60129542198 2026-02-19T13:30:55.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:30:55.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:30:55.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:30:55.917 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:30:55.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:30:55.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:30:55.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:30:55.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:30:55.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:30:56.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:30:56.079 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:30:56.079 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:30:56.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:30:56.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:56.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:30:56.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836543 2026-02-19T13:30:56.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836543 2026-02-19T13:30:56.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836543' 2026-02-19T13:30:56.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:56.151 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:30:56.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705724 2026-02-19T13:30:56.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705724 2026-02-19T13:30:56.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836543 1-38654705724' 2026-02-19T13:30:56.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:56.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:30:56.292 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542202 2026-02-19T13:30:56.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542202 2026-02-19T13:30:56.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836543 1-38654705724 2-60129542202' 2026-02-19T13:30:56.293 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:56.294 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836543 2026-02-19T13:30:56.294 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:56.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:30:56.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836543 2026-02-19T13:30:56.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:56.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836543 2026-02-19T13:30:56.297 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836543 2026-02-19T13:30:56.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836543' 2026-02-19T13:30:56.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:56.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836543 -lt 21474836543 2026-02-19T13:30:56.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:56.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705724 2026-02-19T13:30:56.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:56.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:30:56.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705724 2026-02-19T13:30:56.466 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:56.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705724 2026-02-19T13:30:56.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705724' 2026-02-19T13:30:56.468 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705724 2026-02-19T13:30:56.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:30:56.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705724 -lt 38654705724 2026-02-19T13:30:56.631 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:56.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542202 2026-02-19T13:30:56.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:56.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:30:56.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542202 2026-02-19T13:30:56.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:56.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542202 2026-02-19T13:30:56.635 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542202 2026-02-19T13:30:56.636 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542202' 2026-02-19T13:30:56.636 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:30:56.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542202 -lt 60129542202 2026-02-19T13:30:56.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:30:56.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:30:56.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:30:56.947 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:30:56.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:30:56.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:30:56.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:30:56.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:30:56.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:30:57.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:30:57.116 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:30:57.116 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:30:57.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:30:57.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:57.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:30:57.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836545 2026-02-19T13:30:57.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836545 2026-02-19T13:30:57.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836545' 2026-02-19T13:30:57.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:57.193 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:30:57.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705726 2026-02-19T13:30:57.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705726 2026-02-19T13:30:57.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836545 1-38654705726' 2026-02-19T13:30:57.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:57.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:30:57.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542204 2026-02-19T13:30:57.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542204 2026-02-19T13:30:57.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836545 1-38654705726 2-60129542204' 2026-02-19T13:30:57.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:57.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836545 2026-02-19T13:30:57.337 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:57.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:30:57.339 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836545 2026-02-19T13:30:57.339 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:57.340 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836545 2026-02-19T13:30:57.340 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836545 2026-02-19T13:30:57.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836545' 2026-02-19T13:30:57.341 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:57.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836543 -lt 21474836545 2026-02-19T13:30:57.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:30:58.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:30:58.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:58.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836546 -lt 21474836545 2026-02-19T13:30:58.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:58.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705726 2026-02-19T13:30:58.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:58.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:30:58.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705726 2026-02-19T13:30:58.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:58.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705726 2026-02-19T13:30:58.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705726' 2026-02-19T13:30:58.672 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705726 2026-02-19T13:30:58.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:30:58.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705727 -lt 38654705726 2026-02-19T13:30:58.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:58.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542204 2026-02-19T13:30:58.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:58.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:30:58.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542204 2026-02-19T13:30:58.831 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:58.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542204 2026-02-19T13:30:58.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542204' 2026-02-19T13:30:58.833 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542204 2026-02-19T13:30:58.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:30:59.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542205 -lt 60129542204 2026-02-19T13:30:59.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:30:59.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:30:59.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:30:59.146 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:30:59.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:30:59.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:30:59.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:30:59.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:30:59.156 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:30:59.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:30:59.316 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:30:59.316 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:30:59.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:30:59.317 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:59.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:30:59.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836548 2026-02-19T13:30:59.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836548 2026-02-19T13:30:59.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836548' 2026-02-19T13:30:59.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:59.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:30:59.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705729 2026-02-19T13:30:59.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705729 2026-02-19T13:30:59.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836548 1-38654705729' 2026-02-19T13:30:59.462 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:30:59.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:30:59.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542207 2026-02-19T13:30:59.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542207 2026-02-19T13:30:59.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836548 1-38654705729 2-60129542207' 2026-02-19T13:30:59.532 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:30:59.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836548 2026-02-19T13:30:59.533 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:30:59.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:30:59.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836548 2026-02-19T13:30:59.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:30:59.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836548 2026-02-19T13:30:59.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836548' 2026-02-19T13:30:59.536 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836548 2026-02-19T13:30:59.536 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:30:59.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836546 -lt 21474836548 2026-02-19T13:30:59.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:00.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:00.702 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:00.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836549 -lt 21474836548 2026-02-19T13:31:00.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:00.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705729 2026-02-19T13:31:00.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:00.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:00.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705729 2026-02-19T13:31:00.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:00.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705729 2026-02-19T13:31:00.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705729' 2026-02-19T13:31:00.865 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705729 2026-02-19T13:31:00.865 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:01.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705730 -lt 38654705729 2026-02-19T13:31:01.030 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:01.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542207 2026-02-19T13:31:01.031 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:01.032 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:01.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542207 2026-02-19T13:31:01.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:01.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542207 2026-02-19T13:31:01.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542207' 2026-02-19T13:31:01.034 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542207 2026-02-19T13:31:01.035 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:01.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542208 -lt 60129542207 2026-02-19T13:31:01.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:01.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:01.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:01.343 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:01.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:01.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:01.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:01.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:01.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:01.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:01.504 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:01.504 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:01.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:01.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:01.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:01.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836551 2026-02-19T13:31:01.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836551 2026-02-19T13:31:01.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836551' 2026-02-19T13:31:01.576 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:01.577 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:01.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705733 2026-02-19T13:31:01.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705733 2026-02-19T13:31:01.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836551 1-38654705733' 2026-02-19T13:31:01.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:01.649 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:01.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542210 2026-02-19T13:31:01.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542210 2026-02-19T13:31:01.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836551 1-38654705733 2-60129542210' 2026-02-19T13:31:01.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:01.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836551 2026-02-19T13:31:01.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:01.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:01.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836551 2026-02-19T13:31:01.722 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:01.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836551 2026-02-19T13:31:01.724 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836551 2026-02-19T13:31:01.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836551' 2026-02-19T13:31:01.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:01.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836549 -lt 21474836551 2026-02-19T13:31:01.887 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:02.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:02.888 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:03.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836552 -lt 21474836551 2026-02-19T13:31:03.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:03.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705733 2026-02-19T13:31:03.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:03.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:03.049 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705733 2026-02-19T13:31:03.049 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:03.050 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705733 2026-02-19T13:31:03.051 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705733 2026-02-19T13:31:03.051 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705733' 2026-02-19T13:31:03.051 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:03.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705733 -lt 38654705733 2026-02-19T13:31:03.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:03.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542210 2026-02-19T13:31:03.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:03.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:03.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542210 2026-02-19T13:31:03.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:03.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542210 2026-02-19T13:31:03.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542210' 2026-02-19T13:31:03.221 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542210 2026-02-19T13:31:03.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:03.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542211 -lt 60129542210 2026-02-19T13:31:03.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:03.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:03.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:03.523 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:03.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:03.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:03.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:03.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:03.535 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:03.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:03.696 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:03.696 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:03.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:03.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:03.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:03.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836554 2026-02-19T13:31:03.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836554 2026-02-19T13:31:03.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836554' 2026-02-19T13:31:03.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:03.769 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:03.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705736 2026-02-19T13:31:03.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705736 2026-02-19T13:31:03.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836554 1-38654705736' 2026-02-19T13:31:03.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:03.841 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:03.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542213 2026-02-19T13:31:03.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542213 2026-02-19T13:31:03.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836554 1-38654705736 2-60129542213' 2026-02-19T13:31:03.915 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:03.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836554 2026-02-19T13:31:03.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:03.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:03.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836554 2026-02-19T13:31:03.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:03.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836554 2026-02-19T13:31:03.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836554' 2026-02-19T13:31:03.919 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836554 2026-02-19T13:31:03.919 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:04.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836552 -lt 21474836554 2026-02-19T13:31:04.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:05.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:05.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:05.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836555 -lt 21474836554 2026-02-19T13:31:05.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:05.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705736 2026-02-19T13:31:05.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:05.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:05.236 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705736 2026-02-19T13:31:05.236 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:05.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705736 2026-02-19T13:31:05.238 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705736 2026-02-19T13:31:05.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705736' 2026-02-19T13:31:05.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:05.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705736 -lt 38654705736 2026-02-19T13:31:05.406 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:05.407 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542213 2026-02-19T13:31:05.407 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:05.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:05.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542213 2026-02-19T13:31:05.409 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:05.410 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542213 2026-02-19T13:31:05.411 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542213 2026-02-19T13:31:05.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542213' 2026-02-19T13:31:05.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:05.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542214 -lt 60129542213 2026-02-19T13:31:05.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:05.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:05.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:05.717 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:05.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:05.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:05.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:05.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:05.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:05.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:05.882 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:05.882 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:05.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:05.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:05.882 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:05.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836557 2026-02-19T13:31:05.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836557 2026-02-19T13:31:05.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836557' 2026-02-19T13:31:05.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:05.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:06.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705739 2026-02-19T13:31:06.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705739 2026-02-19T13:31:06.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836557 1-38654705739' 2026-02-19T13:31:06.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:06.027 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:06.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542216 2026-02-19T13:31:06.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542216 2026-02-19T13:31:06.104 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836557 1-38654705739 2-60129542216' 2026-02-19T13:31:06.105 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:06.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836557 2026-02-19T13:31:06.105 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:06.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:06.107 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836557 2026-02-19T13:31:06.107 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:06.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836557 2026-02-19T13:31:06.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836557' 2026-02-19T13:31:06.109 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836557 2026-02-19T13:31:06.109 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:06.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836555 -lt 21474836557 2026-02-19T13:31:06.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:07.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:07.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:07.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836558 -lt 21474836557 2026-02-19T13:31:07.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:07.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705739 2026-02-19T13:31:07.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:07.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:07.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705739 2026-02-19T13:31:07.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:07.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705739 2026-02-19T13:31:07.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705739' 2026-02-19T13:31:07.434 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705739 2026-02-19T13:31:07.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:07.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705739 -lt 38654705739 2026-02-19T13:31:07.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:07.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542216 2026-02-19T13:31:07.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:07.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:07.597 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542216 2026-02-19T13:31:07.597 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:07.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542216 2026-02-19T13:31:07.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542216' 2026-02-19T13:31:07.598 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542216 2026-02-19T13:31:07.599 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:07.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542217 -lt 60129542216 2026-02-19T13:31:07.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:07.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:07.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:07.914 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:07.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:07.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:07.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:07.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:07.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:08.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:08.080 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:08.081 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:08.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:08.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:08.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:08.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836561 2026-02-19T13:31:08.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836561 2026-02-19T13:31:08.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836561' 2026-02-19T13:31:08.152 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:08.153 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:08.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705742 2026-02-19T13:31:08.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705742 2026-02-19T13:31:08.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836561 1-38654705742' 2026-02-19T13:31:08.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:08.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:08.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542220 2026-02-19T13:31:08.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542220 2026-02-19T13:31:08.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836561 1-38654705742 2-60129542220' 2026-02-19T13:31:08.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:08.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836561 2026-02-19T13:31:08.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:08.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:08.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836561 2026-02-19T13:31:08.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:08.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836561 2026-02-19T13:31:08.301 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836561 2026-02-19T13:31:08.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836561' 2026-02-19T13:31:08.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:08.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836561 -lt 21474836561 2026-02-19T13:31:08.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:08.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705742 2026-02-19T13:31:08.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:08.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:08.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705742 2026-02-19T13:31:08.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:08.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705742 2026-02-19T13:31:08.459 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705742 2026-02-19T13:31:08.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705742' 2026-02-19T13:31:08.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:08.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705742 -lt 38654705742 2026-02-19T13:31:08.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:08.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542220 2026-02-19T13:31:08.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:08.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:08.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542220 2026-02-19T13:31:08.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:08.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542220 2026-02-19T13:31:08.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542220' 2026-02-19T13:31:08.622 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542220 2026-02-19T13:31:08.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:08.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542220 -lt 60129542220 2026-02-19T13:31:08.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:08.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:08.793 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:08.938 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:08.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:08.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:08.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:08.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:08.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:09.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:09.106 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:09.106 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:09.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:09.106 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:09.106 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:09.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836563 2026-02-19T13:31:09.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836563 2026-02-19T13:31:09.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836563' 2026-02-19T13:31:09.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:09.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:09.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705744 2026-02-19T13:31:09.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705744 2026-02-19T13:31:09.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836563 1-38654705744' 2026-02-19T13:31:09.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:09.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:09.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542222 2026-02-19T13:31:09.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542222 2026-02-19T13:31:09.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836563 1-38654705744 2-60129542222' 2026-02-19T13:31:09.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:09.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836563 2026-02-19T13:31:09.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:09.326 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:09.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836563 2026-02-19T13:31:09.327 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:09.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836563 2026-02-19T13:31:09.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836563' 2026-02-19T13:31:09.329 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836563 2026-02-19T13:31:09.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:09.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836561 -lt 21474836563 2026-02-19T13:31:09.491 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:10.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:10.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:10.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836564 -lt 21474836563 2026-02-19T13:31:10.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:10.654 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705744 2026-02-19T13:31:10.654 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:10.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:10.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705744 2026-02-19T13:31:10.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:10.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705744 2026-02-19T13:31:10.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705744' 2026-02-19T13:31:10.658 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705744 2026-02-19T13:31:10.658 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:10.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705745 -lt 38654705744 2026-02-19T13:31:10.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:10.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542222 2026-02-19T13:31:10.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:10.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:10.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542222 2026-02-19T13:31:10.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:10.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542222 2026-02-19T13:31:10.830 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542222 2026-02-19T13:31:10.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542222' 2026-02-19T13:31:10.830 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:10.995 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542223 -lt 60129542222 2026-02-19T13:31:10.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:10.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:10.996 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:11.142 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:11.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:11.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:11.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:11.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:11.153 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:11.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:11.312 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:11.312 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:11.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:11.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:11.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:11.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836566 2026-02-19T13:31:11.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836566 2026-02-19T13:31:11.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836566' 2026-02-19T13:31:11.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:11.385 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:11.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705747 2026-02-19T13:31:11.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705747 2026-02-19T13:31:11.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836566 1-38654705747' 2026-02-19T13:31:11.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:11.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:11.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542225 2026-02-19T13:31:11.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542225 2026-02-19T13:31:11.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836566 1-38654705747 2-60129542225' 2026-02-19T13:31:11.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:11.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836566 2026-02-19T13:31:11.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:11.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:11.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836566 2026-02-19T13:31:11.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:11.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836566 2026-02-19T13:31:11.534 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836566 2026-02-19T13:31:11.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836566' 2026-02-19T13:31:11.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:11.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836564 -lt 21474836566 2026-02-19T13:31:11.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:12.703 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:12.704 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:12.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836567 -lt 21474836566 2026-02-19T13:31:12.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:12.862 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705747 2026-02-19T13:31:12.862 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:12.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:12.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705747 2026-02-19T13:31:12.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:12.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705747 2026-02-19T13:31:12.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705747' 2026-02-19T13:31:12.866 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705747 2026-02-19T13:31:12.866 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:13.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705748 -lt 38654705747 2026-02-19T13:31:13.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:13.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542225 2026-02-19T13:31:13.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:13.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:13.039 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542225 2026-02-19T13:31:13.039 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:13.040 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542225 2026-02-19T13:31:13.041 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542225 2026-02-19T13:31:13.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542225' 2026-02-19T13:31:13.041 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:13.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542226 -lt 60129542225 2026-02-19T13:31:13.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:13.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:13.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:13.353 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:13.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:13.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:13.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:13.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:13.364 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:13.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:13.525 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:13.525 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:13.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:13.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:13.525 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:13.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836569 2026-02-19T13:31:13.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836569 2026-02-19T13:31:13.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836569' 2026-02-19T13:31:13.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:13.598 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:13.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705751 2026-02-19T13:31:13.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705751 2026-02-19T13:31:13.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836569 1-38654705751' 2026-02-19T13:31:13.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:13.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:13.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542228 2026-02-19T13:31:13.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542228 2026-02-19T13:31:13.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836569 1-38654705751 2-60129542228' 2026-02-19T13:31:13.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:13.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836569 2026-02-19T13:31:13.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:13.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:13.744 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836569 2026-02-19T13:31:13.744 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:13.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836569 2026-02-19T13:31:13.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836569' 2026-02-19T13:31:13.746 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836569 2026-02-19T13:31:13.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:13.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836567 -lt 21474836569 2026-02-19T13:31:13.911 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:14.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:14.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:15.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836570 -lt 21474836569 2026-02-19T13:31:15.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:15.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705751 2026-02-19T13:31:15.075 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:15.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:15.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705751 2026-02-19T13:31:15.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:15.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705751 2026-02-19T13:31:15.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705751' 2026-02-19T13:31:15.079 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705751 2026-02-19T13:31:15.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:15.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705751 -lt 38654705751 2026-02-19T13:31:15.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:15.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542228 2026-02-19T13:31:15.239 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:15.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:15.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542228 2026-02-19T13:31:15.240 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:15.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542228 2026-02-19T13:31:15.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542228' 2026-02-19T13:31:15.242 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542228 2026-02-19T13:31:15.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:15.408 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542229 -lt 60129542228 2026-02-19T13:31:15.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:15.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:15.409 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:15.558 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:15.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:15.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:15.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:15.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:15.568 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:15.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:15.725 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:15.725 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:15.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:15.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:15.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:15.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836572 2026-02-19T13:31:15.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836572 2026-02-19T13:31:15.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836572' 2026-02-19T13:31:15.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:15.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:15.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705754 2026-02-19T13:31:15.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705754 2026-02-19T13:31:15.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836572 1-38654705754' 2026-02-19T13:31:15.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:15.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:15.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542231 2026-02-19T13:31:15.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542231 2026-02-19T13:31:15.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836572 1-38654705754 2-60129542231' 2026-02-19T13:31:15.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:15.942 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836572 2026-02-19T13:31:15.942 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:15.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:15.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836572 2026-02-19T13:31:15.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:15.945 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836572 2026-02-19T13:31:15.946 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836572 2026-02-19T13:31:15.946 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836572' 2026-02-19T13:31:15.946 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:16.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836570 -lt 21474836572 2026-02-19T13:31:16.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:17.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:17.117 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:17.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836573 -lt 21474836572 2026-02-19T13:31:17.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:17.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705754 2026-02-19T13:31:17.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:17.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:17.277 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705754 2026-02-19T13:31:17.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:17.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705754 2026-02-19T13:31:17.279 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705754 2026-02-19T13:31:17.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705754' 2026-02-19T13:31:17.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:17.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705754 -lt 38654705754 2026-02-19T13:31:17.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:17.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542231 2026-02-19T13:31:17.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:17.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:17.441 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542231 2026-02-19T13:31:17.441 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:17.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542231 2026-02-19T13:31:17.443 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542231 2026-02-19T13:31:17.443 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542231' 2026-02-19T13:31:17.443 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:17.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542232 -lt 60129542231 2026-02-19T13:31:17.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:17.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:17.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:17.749 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:17.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:17.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:17.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:17.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:17.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:17.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:17.919 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:17.919 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:17.919 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:17.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:17.920 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:17.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836575 2026-02-19T13:31:17.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836575 2026-02-19T13:31:17.991 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836575' 2026-02-19T13:31:17.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:17.992 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:18.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705757 2026-02-19T13:31:18.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705757 2026-02-19T13:31:18.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836575 1-38654705757' 2026-02-19T13:31:18.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:18.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:18.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542234 2026-02-19T13:31:18.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542234 2026-02-19T13:31:18.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836575 1-38654705757 2-60129542234' 2026-02-19T13:31:18.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:18.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836575 2026-02-19T13:31:18.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:18.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:18.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836575 2026-02-19T13:31:18.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:18.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836575 2026-02-19T13:31:18.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836575' 2026-02-19T13:31:18.140 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836575 2026-02-19T13:31:18.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:18.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836573 -lt 21474836575 2026-02-19T13:31:18.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:19.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:19.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:19.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836576 -lt 21474836575 2026-02-19T13:31:19.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:19.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705757 2026-02-19T13:31:19.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:19.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:19.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705757 2026-02-19T13:31:19.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:19.463 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705757 2026-02-19T13:31:19.463 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705757 2026-02-19T13:31:19.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705757' 2026-02-19T13:31:19.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:19.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705757 -lt 38654705757 2026-02-19T13:31:19.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:19.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542234 2026-02-19T13:31:19.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:19.626 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:19.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542234 2026-02-19T13:31:19.627 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:19.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542234 2026-02-19T13:31:19.629 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542234' 2026-02-19T13:31:19.629 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542234 2026-02-19T13:31:19.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:19.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542235 -lt 60129542234 2026-02-19T13:31:19.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:19.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:19.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:19.948 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:19.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:19.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:19.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:19.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:19.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:20.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:20.111 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:20.111 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:20.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:20.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:20.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:20.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836579 2026-02-19T13:31:20.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836579 2026-02-19T13:31:20.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836579' 2026-02-19T13:31:20.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:20.182 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:20.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705760 2026-02-19T13:31:20.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705760 2026-02-19T13:31:20.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836579 1-38654705760' 2026-02-19T13:31:20.253 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:20.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:20.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542238 2026-02-19T13:31:20.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542238 2026-02-19T13:31:20.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836579 1-38654705760 2-60129542238' 2026-02-19T13:31:20.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:20.328 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836579 2026-02-19T13:31:20.328 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:20.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:20.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836579 2026-02-19T13:31:20.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:20.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836579 2026-02-19T13:31:20.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836579' 2026-02-19T13:31:20.332 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836579 2026-02-19T13:31:20.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:20.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836579 -lt 21474836579 2026-02-19T13:31:20.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:20.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705760 2026-02-19T13:31:20.494 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:20.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:20.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705760 2026-02-19T13:31:20.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:20.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705760 2026-02-19T13:31:20.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705760' 2026-02-19T13:31:20.497 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705760 2026-02-19T13:31:20.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:20.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705760 -lt 38654705760 2026-02-19T13:31:20.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:20.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542238 2026-02-19T13:31:20.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:20.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:20.657 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542238 2026-02-19T13:31:20.657 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:20.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542238 2026-02-19T13:31:20.659 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542238 2026-02-19T13:31:20.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542238' 2026-02-19T13:31:20.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:20.816 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542238 -lt 60129542238 2026-02-19T13:31:20.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:20.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:20.817 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:20.965 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:20.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:20.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:20.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:20.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:20.975 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:21.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:21.131 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:21.131 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:21.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:21.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:21.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:21.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836581 2026-02-19T13:31:21.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836581 2026-02-19T13:31:21.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836581' 2026-02-19T13:31:21.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:21.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:21.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705762 2026-02-19T13:31:21.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705762 2026-02-19T13:31:21.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836581 1-38654705762' 2026-02-19T13:31:21.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:21.274 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:21.344 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542240 2026-02-19T13:31:21.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542240 2026-02-19T13:31:21.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836581 1-38654705762 2-60129542240' 2026-02-19T13:31:21.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:21.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836581 2026-02-19T13:31:21.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:21.346 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:21.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836581 2026-02-19T13:31:21.347 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:21.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836581 2026-02-19T13:31:21.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836581' 2026-02-19T13:31:21.349 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836581 2026-02-19T13:31:21.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:21.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836579 -lt 21474836581 2026-02-19T13:31:21.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:22.506 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:22.507 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:22.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836582 -lt 21474836581 2026-02-19T13:31:22.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:22.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705762 2026-02-19T13:31:22.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:22.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:22.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705762 2026-02-19T13:31:22.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:22.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705762 2026-02-19T13:31:22.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705762' 2026-02-19T13:31:22.670 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705762 2026-02-19T13:31:22.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:22.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705763 -lt 38654705762 2026-02-19T13:31:22.836 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:22.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542240 2026-02-19T13:31:22.837 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:22.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:22.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542240 2026-02-19T13:31:22.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:22.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542240 2026-02-19T13:31:22.840 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542240 2026-02-19T13:31:22.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542240' 2026-02-19T13:31:22.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:22.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542241 -lt 60129542240 2026-02-19T13:31:22.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:22.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:22.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:23.145 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:23.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:23.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:23.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:23.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:23.155 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:23.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:23.308 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:23.308 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:23.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:23.308 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:23.308 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:23.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836584 2026-02-19T13:31:23.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836584 2026-02-19T13:31:23.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836584' 2026-02-19T13:31:23.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:23.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:23.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705765 2026-02-19T13:31:23.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705765 2026-02-19T13:31:23.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836584 1-38654705765' 2026-02-19T13:31:23.453 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:23.454 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:23.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542243 2026-02-19T13:31:23.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542243 2026-02-19T13:31:23.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836584 1-38654705765 2-60129542243' 2026-02-19T13:31:23.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:23.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836584 2026-02-19T13:31:23.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:23.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:23.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836584 2026-02-19T13:31:23.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:23.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836584 2026-02-19T13:31:23.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836584' 2026-02-19T13:31:23.534 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836584 2026-02-19T13:31:23.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:23.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836582 -lt 21474836584 2026-02-19T13:31:23.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:24.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:24.691 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:24.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836585 -lt 21474836584 2026-02-19T13:31:24.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:24.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705765 2026-02-19T13:31:24.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:24.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:24.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705765 2026-02-19T13:31:24.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:24.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705765 2026-02-19T13:31:24.854 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705765 2026-02-19T13:31:24.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705765' 2026-02-19T13:31:24.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:25.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705766 -lt 38654705765 2026-02-19T13:31:25.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:25.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542243 2026-02-19T13:31:25.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:25.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:25.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542243 2026-02-19T13:31:25.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:25.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542243 2026-02-19T13:31:25.024 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542243 2026-02-19T13:31:25.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542243' 2026-02-19T13:31:25.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:25.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542244 -lt 60129542243 2026-02-19T13:31:25.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:25.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:25.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:25.343 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:25.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:25.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:25.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:25.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:25.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:25.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:25.513 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:25.513 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:25.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:25.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:25.513 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:25.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836587 2026-02-19T13:31:25.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836587 2026-02-19T13:31:25.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836587' 2026-02-19T13:31:25.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:25.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:25.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705769 2026-02-19T13:31:25.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705769 2026-02-19T13:31:25.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836587 1-38654705769' 2026-02-19T13:31:25.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:25.658 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:25.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542246 2026-02-19T13:31:25.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542246 2026-02-19T13:31:25.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836587 1-38654705769 2-60129542246' 2026-02-19T13:31:25.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:25.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836587 2026-02-19T13:31:25.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:25.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:25.732 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836587 2026-02-19T13:31:25.732 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:25.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836587 2026-02-19T13:31:25.734 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836587' 2026-02-19T13:31:25.734 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836587 2026-02-19T13:31:25.734 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:25.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836585 -lt 21474836587 2026-02-19T13:31:25.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:26.902 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:26.903 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:27.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836588 -lt 21474836587 2026-02-19T13:31:27.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:27.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705769 2026-02-19T13:31:27.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:27.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:27.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705769 2026-02-19T13:31:27.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:27.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705769 2026-02-19T13:31:27.063 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705769 2026-02-19T13:31:27.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705769' 2026-02-19T13:31:27.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:27.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705769 -lt 38654705769 2026-02-19T13:31:27.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:27.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542246 2026-02-19T13:31:27.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:27.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:27.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542246 2026-02-19T13:31:27.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:27.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542246 2026-02-19T13:31:27.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542246' 2026-02-19T13:31:27.226 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542246 2026-02-19T13:31:27.227 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:27.391 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542247 -lt 60129542246 2026-02-19T13:31:27.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:27.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:27.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:27.539 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:27.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:27.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:27.550 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:27.551 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:27.551 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:27.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:27.712 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:27.712 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:27.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:27.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:27.713 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:27.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836590 2026-02-19T13:31:27.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836590 2026-02-19T13:31:27.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836590' 2026-02-19T13:31:27.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:27.786 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:27.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705772 2026-02-19T13:31:27.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705772 2026-02-19T13:31:27.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836590 1-38654705772' 2026-02-19T13:31:27.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:27.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:27.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542249 2026-02-19T13:31:27.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542249 2026-02-19T13:31:27.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836590 1-38654705772 2-60129542249' 2026-02-19T13:31:27.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:27.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836590 2026-02-19T13:31:27.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:27.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:27.937 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836590 2026-02-19T13:31:27.937 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:27.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836590 2026-02-19T13:31:27.939 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836590 2026-02-19T13:31:27.939 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836590' 2026-02-19T13:31:27.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:28.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836588 -lt 21474836590 2026-02-19T13:31:28.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:29.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:29.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:29.255 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836591 -lt 21474836590 2026-02-19T13:31:29.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:29.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705772 2026-02-19T13:31:29.256 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:29.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:29.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705772 2026-02-19T13:31:29.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:29.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705772 2026-02-19T13:31:29.260 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705772 2026-02-19T13:31:29.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705772' 2026-02-19T13:31:29.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:29.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705772 -lt 38654705772 2026-02-19T13:31:29.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:29.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542249 2026-02-19T13:31:29.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:29.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:29.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542249 2026-02-19T13:31:29.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:29.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542249 2026-02-19T13:31:29.433 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542249 2026-02-19T13:31:29.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542249' 2026-02-19T13:31:29.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:29.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542250 -lt 60129542249 2026-02-19T13:31:29.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:29.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:29.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:29.752 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:29.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:29.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:29.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:29.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:29.762 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:29.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:29.916 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:29.916 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:29.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:29.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:29.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:29.989 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836593 2026-02-19T13:31:29.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836593 2026-02-19T13:31:29.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836593' 2026-02-19T13:31:29.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:29.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:30.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705775 2026-02-19T13:31:30.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705775 2026-02-19T13:31:30.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836593 1-38654705775' 2026-02-19T13:31:30.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:30.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:30.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542252 2026-02-19T13:31:30.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542252 2026-02-19T13:31:30.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836593 1-38654705775 2-60129542252' 2026-02-19T13:31:30.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:30.136 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836593 2026-02-19T13:31:30.136 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:30.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:30.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836593 2026-02-19T13:31:30.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:30.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836593 2026-02-19T13:31:30.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836593' 2026-02-19T13:31:30.139 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836593 2026-02-19T13:31:30.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:30.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836591 -lt 21474836593 2026-02-19T13:31:30.297 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:31.298 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:31.299 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:31.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836594 -lt 21474836593 2026-02-19T13:31:31.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:31.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705775 2026-02-19T13:31:31.457 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:31.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:31.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705775 2026-02-19T13:31:31.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:31.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705775 2026-02-19T13:31:31.460 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705775 2026-02-19T13:31:31.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705775' 2026-02-19T13:31:31.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:31.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705775 -lt 38654705775 2026-02-19T13:31:31.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:31.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542252 2026-02-19T13:31:31.629 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:31.630 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:31.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542252 2026-02-19T13:31:31.631 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:31.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542252 2026-02-19T13:31:31.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542252' 2026-02-19T13:31:31.633 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542252 2026-02-19T13:31:31.633 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:31.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542253 -lt 60129542252 2026-02-19T13:31:31.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:31.802 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:31.803 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:31.948 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:31.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:31.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:31.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:31.958 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:31.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:32.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:32.112 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:32.113 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:32.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:32.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:32.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:32.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836597 2026-02-19T13:31:32.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836597 2026-02-19T13:31:32.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836597' 2026-02-19T13:31:32.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:32.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:32.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705778 2026-02-19T13:31:32.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705778 2026-02-19T13:31:32.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836597 1-38654705778' 2026-02-19T13:31:32.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:32.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:32.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542256 2026-02-19T13:31:32.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542256 2026-02-19T13:31:32.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836597 1-38654705778 2-60129542256' 2026-02-19T13:31:32.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:32.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836597 2026-02-19T13:31:32.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:32.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:32.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836597 2026-02-19T13:31:32.333 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:32.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836597 2026-02-19T13:31:32.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836597' 2026-02-19T13:31:32.335 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836597 2026-02-19T13:31:32.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:32.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836597 -lt 21474836597 2026-02-19T13:31:32.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:32.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705778 2026-02-19T13:31:32.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:32.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:32.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705778 2026-02-19T13:31:32.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:32.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705778 2026-02-19T13:31:32.497 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705778 2026-02-19T13:31:32.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705778' 2026-02-19T13:31:32.497 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:32.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705778 -lt 38654705778 2026-02-19T13:31:32.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:32.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542256 2026-02-19T13:31:32.664 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:32.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:32.665 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542256 2026-02-19T13:31:32.665 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:32.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542256 2026-02-19T13:31:32.667 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542256 2026-02-19T13:31:32.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542256' 2026-02-19T13:31:32.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:32.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542256 -lt 60129542256 2026-02-19T13:31:32.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:32.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:32.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:32.976 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:32.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:32.987 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:32.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:32.988 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:32.988 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:33.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:33.146 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:33.147 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:33.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:33.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:33.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:33.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836599 2026-02-19T13:31:33.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836599 2026-02-19T13:31:33.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836599' 2026-02-19T13:31:33.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:33.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:33.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705780 2026-02-19T13:31:33.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705780 2026-02-19T13:31:33.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836599 1-38654705780' 2026-02-19T13:31:33.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:33.290 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:33.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542258 2026-02-19T13:31:33.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542258 2026-02-19T13:31:33.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836599 1-38654705780 2-60129542258' 2026-02-19T13:31:33.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:33.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836599 2026-02-19T13:31:33.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:33.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:33.363 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836599 2026-02-19T13:31:33.363 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:33.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836599 2026-02-19T13:31:33.365 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836599 2026-02-19T13:31:33.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836599' 2026-02-19T13:31:33.365 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:33.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836597 -lt 21474836599 2026-02-19T13:31:33.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:34.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:34.523 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:34.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836600 -lt 21474836599 2026-02-19T13:31:34.680 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:34.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705780 2026-02-19T13:31:34.681 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:34.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:34.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705780 2026-02-19T13:31:34.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:34.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705780 2026-02-19T13:31:34.685 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705780 2026-02-19T13:31:34.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705780' 2026-02-19T13:31:34.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:34.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705781 -lt 38654705780 2026-02-19T13:31:34.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:34.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542258 2026-02-19T13:31:34.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:34.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:34.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542258 2026-02-19T13:31:34.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:34.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542258 2026-02-19T13:31:34.856 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542258 2026-02-19T13:31:34.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542258' 2026-02-19T13:31:34.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:35.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542259 -lt 60129542258 2026-02-19T13:31:35.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:35.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:35.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:35.165 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:35.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:35.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:35.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:35.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:35.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:35.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:35.332 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:35.332 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:35.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:35.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:35.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:35.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836602 2026-02-19T13:31:35.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836602 2026-02-19T13:31:35.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836602' 2026-02-19T13:31:35.402 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:35.402 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:35.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705783 2026-02-19T13:31:35.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705783 2026-02-19T13:31:35.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836602 1-38654705783' 2026-02-19T13:31:35.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:35.472 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:35.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542261 2026-02-19T13:31:35.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542261 2026-02-19T13:31:35.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836602 1-38654705783 2-60129542261' 2026-02-19T13:31:35.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:35.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836602 2026-02-19T13:31:35.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:35.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:35.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836602 2026-02-19T13:31:35.547 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:35.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836602 2026-02-19T13:31:35.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836602' 2026-02-19T13:31:35.549 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836602 2026-02-19T13:31:35.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:35.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836600 -lt 21474836602 2026-02-19T13:31:35.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:36.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:36.714 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:36.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836603 -lt 21474836602 2026-02-19T13:31:36.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:36.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705783 2026-02-19T13:31:36.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:36.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:36.877 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705783 2026-02-19T13:31:36.877 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:36.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705783 2026-02-19T13:31:36.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705783' 2026-02-19T13:31:36.879 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705783 2026-02-19T13:31:36.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:37.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705784 -lt 38654705783 2026-02-19T13:31:37.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:37.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542261 2026-02-19T13:31:37.034 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:37.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:37.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542261 2026-02-19T13:31:37.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:37.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542261 2026-02-19T13:31:37.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542261' 2026-02-19T13:31:37.038 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542261 2026-02-19T13:31:37.038 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:37.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542262 -lt 60129542261 2026-02-19T13:31:37.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:37.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:37.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:37.351 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:37.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:37.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:37.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:37.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:37.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:37.511 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:37.512 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:37.512 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:37.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:37.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:37.512 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:37.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836605 2026-02-19T13:31:37.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836605 2026-02-19T13:31:37.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836605' 2026-02-19T13:31:37.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:37.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:37.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705787 2026-02-19T13:31:37.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705787 2026-02-19T13:31:37.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836605 1-38654705787' 2026-02-19T13:31:37.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:37.658 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:37.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542264 2026-02-19T13:31:37.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542264 2026-02-19T13:31:37.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836605 1-38654705787 2-60129542264' 2026-02-19T13:31:37.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:37.729 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836605 2026-02-19T13:31:37.729 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:37.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:37.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836605 2026-02-19T13:31:37.731 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:37.732 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836605 2026-02-19T13:31:37.733 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836605 2026-02-19T13:31:37.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836605' 2026-02-19T13:31:37.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:37.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836603 -lt 21474836605 2026-02-19T13:31:37.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:38.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:38.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:39.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836606 -lt 21474836605 2026-02-19T13:31:39.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:39.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705787 2026-02-19T13:31:39.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:39.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:39.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705787 2026-02-19T13:31:39.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:39.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705787 2026-02-19T13:31:39.062 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705787 2026-02-19T13:31:39.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705787' 2026-02-19T13:31:39.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:39.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705787 -lt 38654705787 2026-02-19T13:31:39.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:39.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542264 2026-02-19T13:31:39.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:39.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:39.236 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542264 2026-02-19T13:31:39.236 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:39.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542264 2026-02-19T13:31:39.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542264' 2026-02-19T13:31:39.238 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542264 2026-02-19T13:31:39.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:39.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542265 -lt 60129542264 2026-02-19T13:31:39.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:39.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:39.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:39.547 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:39.559 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:39.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:39.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:39.560 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:39.560 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:39.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:39.718 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:39.718 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:39.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:39.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:39.719 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:39.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836608 2026-02-19T13:31:39.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836608 2026-02-19T13:31:39.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836608' 2026-02-19T13:31:39.789 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:39.789 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:39.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705790 2026-02-19T13:31:39.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705790 2026-02-19T13:31:39.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836608 1-38654705790' 2026-02-19T13:31:39.863 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:39.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:39.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542267 2026-02-19T13:31:39.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542267 2026-02-19T13:31:39.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836608 1-38654705790 2-60129542267' 2026-02-19T13:31:39.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:39.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836608 2026-02-19T13:31:39.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:39.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:39.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836608 2026-02-19T13:31:39.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:39.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836608 2026-02-19T13:31:39.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836608' 2026-02-19T13:31:39.940 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836608 2026-02-19T13:31:39.940 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:40.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836606 -lt 21474836608 2026-02-19T13:31:40.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:41.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:41.101 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:41.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836609 -lt 21474836608 2026-02-19T13:31:41.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:41.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705790 2026-02-19T13:31:41.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:41.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:41.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705790 2026-02-19T13:31:41.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:41.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705790 2026-02-19T13:31:41.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705790' 2026-02-19T13:31:41.265 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705790 2026-02-19T13:31:41.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:41.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705790 -lt 38654705790 2026-02-19T13:31:41.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:41.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542267 2026-02-19T13:31:41.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:41.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:41.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542267 2026-02-19T13:31:41.434 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:41.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542267 2026-02-19T13:31:41.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542267' 2026-02-19T13:31:41.436 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542267 2026-02-19T13:31:41.437 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:41.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542268 -lt 60129542267 2026-02-19T13:31:41.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:41.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:41.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:41.743 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:41.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:41.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:41.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:41.752 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:41.753 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:41.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:41.907 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:41.907 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:41.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:41.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:41.908 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:41.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836611 2026-02-19T13:31:41.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836611 2026-02-19T13:31:41.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836611' 2026-02-19T13:31:41.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:41.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:42.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705793 2026-02-19T13:31:42.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705793 2026-02-19T13:31:42.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836611 1-38654705793' 2026-02-19T13:31:42.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:42.054 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:42.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542270 2026-02-19T13:31:42.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542270 2026-02-19T13:31:42.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836611 1-38654705793 2-60129542270' 2026-02-19T13:31:42.125 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:42.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836611 2026-02-19T13:31:42.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:42.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:42.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836611 2026-02-19T13:31:42.127 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:42.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836611 2026-02-19T13:31:42.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836611' 2026-02-19T13:31:42.129 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836611 2026-02-19T13:31:42.129 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:42.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836609 -lt 21474836611 2026-02-19T13:31:42.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:43.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:43.287 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:43.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836612 -lt 21474836611 2026-02-19T13:31:43.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:43.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705793 2026-02-19T13:31:43.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:43.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:43.452 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705793 2026-02-19T13:31:43.452 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:43.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705793 2026-02-19T13:31:43.454 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705793 2026-02-19T13:31:43.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705793' 2026-02-19T13:31:43.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:43.623 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705793 -lt 38654705793 2026-02-19T13:31:43.624 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:43.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542270 2026-02-19T13:31:43.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:43.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:43.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542270 2026-02-19T13:31:43.626 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:43.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542270 2026-02-19T13:31:43.628 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542270 2026-02-19T13:31:43.628 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542270' 2026-02-19T13:31:43.628 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:43.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542271 -lt 60129542270 2026-02-19T13:31:43.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:43.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:43.792 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:43.945 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:43.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:43.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:43.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:43.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:43.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:44.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:44.113 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:44.113 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:44.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:44.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:44.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:44.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836615 2026-02-19T13:31:44.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836615 2026-02-19T13:31:44.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836615' 2026-02-19T13:31:44.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:44.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:44.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705796 2026-02-19T13:31:44.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705796 2026-02-19T13:31:44.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836615 1-38654705796' 2026-02-19T13:31:44.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:44.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:44.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542274 2026-02-19T13:31:44.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542274 2026-02-19T13:31:44.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836615 1-38654705796 2-60129542274' 2026-02-19T13:31:44.330 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:44.331 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836615 2026-02-19T13:31:44.331 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:44.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:44.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836615 2026-02-19T13:31:44.332 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:44.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836615 2026-02-19T13:31:44.334 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836615 2026-02-19T13:31:44.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836615' 2026-02-19T13:31:44.334 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:44.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836615 -lt 21474836615 2026-02-19T13:31:44.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:44.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705796 2026-02-19T13:31:44.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:44.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:44.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705796 2026-02-19T13:31:44.506 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:44.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705796 2026-02-19T13:31:44.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705796' 2026-02-19T13:31:44.508 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705796 2026-02-19T13:31:44.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:44.664 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705796 -lt 38654705796 2026-02-19T13:31:44.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:44.665 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542274 2026-02-19T13:31:44.665 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:44.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:44.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542274 2026-02-19T13:31:44.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:44.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542274 2026-02-19T13:31:44.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542274' 2026-02-19T13:31:44.669 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542274 2026-02-19T13:31:44.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:44.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542274 -lt 60129542274 2026-02-19T13:31:44.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:44.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:44.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:44.984 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:44.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:44.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:44.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:44.994 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:44.995 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:45.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:45.147 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:45.147 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:45.147 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:45.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:45.148 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:45.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836617 2026-02-19T13:31:45.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836617 2026-02-19T13:31:45.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836617' 2026-02-19T13:31:45.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:45.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:45.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705798 2026-02-19T13:31:45.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705798 2026-02-19T13:31:45.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836617 1-38654705798' 2026-02-19T13:31:45.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:45.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:45.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542276 2026-02-19T13:31:45.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542276 2026-02-19T13:31:45.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836617 1-38654705798 2-60129542276' 2026-02-19T13:31:45.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:45.365 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836617 2026-02-19T13:31:45.365 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:45.366 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:45.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836617 2026-02-19T13:31:45.367 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:45.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836617 2026-02-19T13:31:45.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836617' 2026-02-19T13:31:45.369 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836617 2026-02-19T13:31:45.369 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:45.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836615 -lt 21474836617 2026-02-19T13:31:45.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:46.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:46.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:46.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836618 -lt 21474836617 2026-02-19T13:31:46.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:46.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705798 2026-02-19T13:31:46.683 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:46.684 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:46.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705798 2026-02-19T13:31:46.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:46.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705798 2026-02-19T13:31:46.686 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705798 2026-02-19T13:31:46.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705798' 2026-02-19T13:31:46.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:46.839 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705799 -lt 38654705798 2026-02-19T13:31:46.840 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:46.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542276 2026-02-19T13:31:46.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:46.841 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:46.842 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542276 2026-02-19T13:31:46.842 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:46.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542276 2026-02-19T13:31:46.844 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542276 2026-02-19T13:31:46.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542276' 2026-02-19T13:31:46.844 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:47.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542277 -lt 60129542276 2026-02-19T13:31:47.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:47.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:47.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:47.151 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:47.162 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:47.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:47.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:47.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:47.163 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:47.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:47.319 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:47.320 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:47.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:47.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:47.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:47.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836620 2026-02-19T13:31:47.393 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836620 2026-02-19T13:31:47.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836620' 2026-02-19T13:31:47.394 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:47.394 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:47.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705801 2026-02-19T13:31:47.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705801 2026-02-19T13:31:47.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836620 1-38654705801' 2026-02-19T13:31:47.465 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:47.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:47.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542279 2026-02-19T13:31:47.536 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542279 2026-02-19T13:31:47.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836620 1-38654705801 2-60129542279' 2026-02-19T13:31:47.537 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:47.537 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836620 2026-02-19T13:31:47.537 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:47.538 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:47.539 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836620 2026-02-19T13:31:47.539 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:47.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836620 2026-02-19T13:31:47.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836620' 2026-02-19T13:31:47.541 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836620 2026-02-19T13:31:47.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:47.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836618 -lt 21474836620 2026-02-19T13:31:47.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:48.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:48.712 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:48.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836621 -lt 21474836620 2026-02-19T13:31:48.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:48.868 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705801 2026-02-19T13:31:48.869 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:48.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:48.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705801 2026-02-19T13:31:48.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:48.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705801 2026-02-19T13:31:48.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705801' 2026-02-19T13:31:48.872 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705801 2026-02-19T13:31:48.873 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:49.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705802 -lt 38654705801 2026-02-19T13:31:49.031 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:49.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542279 2026-02-19T13:31:49.032 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:49.033 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:49.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542279 2026-02-19T13:31:49.033 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:49.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542279 2026-02-19T13:31:49.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542279' 2026-02-19T13:31:49.035 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542279 2026-02-19T13:31:49.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:49.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542280 -lt 60129542279 2026-02-19T13:31:49.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:49.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:49.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:49.337 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:49.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:49.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:49.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:49.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:49.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:49.507 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:49.507 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:49.508 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:49.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:49.508 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:49.508 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:49.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836623 2026-02-19T13:31:49.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836623 2026-02-19T13:31:49.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836623' 2026-02-19T13:31:49.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:49.581 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:49.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705805 2026-02-19T13:31:49.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705805 2026-02-19T13:31:49.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836623 1-38654705805' 2026-02-19T13:31:49.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:49.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:49.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542282 2026-02-19T13:31:49.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542282 2026-02-19T13:31:49.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836623 1-38654705805 2-60129542282' 2026-02-19T13:31:49.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:49.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836623 2026-02-19T13:31:49.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:49.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:49.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836623 2026-02-19T13:31:49.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:49.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836623 2026-02-19T13:31:49.729 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836623 2026-02-19T13:31:49.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836623' 2026-02-19T13:31:49.729 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:49.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836621 -lt 21474836623 2026-02-19T13:31:49.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:50.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:50.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:51.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836624 -lt 21474836623 2026-02-19T13:31:51.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:51.055 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705805 2026-02-19T13:31:51.055 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:51.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:51.056 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705805 2026-02-19T13:31:51.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:51.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705805 2026-02-19T13:31:51.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705805' 2026-02-19T13:31:51.059 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705805 2026-02-19T13:31:51.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:51.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705805 -lt 38654705805 2026-02-19T13:31:51.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:51.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542282 2026-02-19T13:31:51.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:51.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:51.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542282 2026-02-19T13:31:51.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:51.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542282 2026-02-19T13:31:51.227 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542282 2026-02-19T13:31:51.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542282' 2026-02-19T13:31:51.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:51.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542283 -lt 60129542282 2026-02-19T13:31:51.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:51.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:51.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:51.546 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:51.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:51.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:51.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:51.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:51.557 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:51.713 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:51.713 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:51.714 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:51.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:51.714 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:51.714 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:51.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836626 2026-02-19T13:31:51.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836626 2026-02-19T13:31:51.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836626' 2026-02-19T13:31:51.786 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:51.786 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:51.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705808 2026-02-19T13:31:51.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705808 2026-02-19T13:31:51.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836626 1-38654705808' 2026-02-19T13:31:51.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:51.856 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:51.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542285 2026-02-19T13:31:51.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542285 2026-02-19T13:31:51.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836626 1-38654705808 2-60129542285' 2026-02-19T13:31:51.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:51.928 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836626 2026-02-19T13:31:51.929 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:51.930 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:51.931 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836626 2026-02-19T13:31:51.931 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:51.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836626 2026-02-19T13:31:51.932 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836626 2026-02-19T13:31:51.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836626' 2026-02-19T13:31:51.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:52.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836624 -lt 21474836626 2026-02-19T13:31:52.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:53.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:53.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:53.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836627 -lt 21474836626 2026-02-19T13:31:53.257 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:53.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705808 2026-02-19T13:31:53.258 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:53.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:53.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705808 2026-02-19T13:31:53.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:53.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705808 2026-02-19T13:31:53.261 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705808 2026-02-19T13:31:53.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705808' 2026-02-19T13:31:53.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:53.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705808 -lt 38654705808 2026-02-19T13:31:53.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:53.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542285 2026-02-19T13:31:53.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:53.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:53.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542285 2026-02-19T13:31:53.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:53.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542285 2026-02-19T13:31:53.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542285' 2026-02-19T13:31:53.434 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542285 2026-02-19T13:31:53.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:53.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542286 -lt 60129542285 2026-02-19T13:31:53.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:53.606 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:53.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:53.752 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:53.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:53.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:53.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:53.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:53.763 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:53.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:53.916 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:53.916 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:53.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:53.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:53.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:53.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836629 2026-02-19T13:31:53.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836629 2026-02-19T13:31:53.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836629' 2026-02-19T13:31:53.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:53.990 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:54.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705811 2026-02-19T13:31:54.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705811 2026-02-19T13:31:54.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836629 1-38654705811' 2026-02-19T13:31:54.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:54.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:54.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542288 2026-02-19T13:31:54.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542288 2026-02-19T13:31:54.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836629 1-38654705811 2-60129542288' 2026-02-19T13:31:54.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:54.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836629 2026-02-19T13:31:54.134 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:54.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:54.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836629 2026-02-19T13:31:54.136 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:54.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836629 2026-02-19T13:31:54.137 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836629 2026-02-19T13:31:54.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836629' 2026-02-19T13:31:54.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:54.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836627 -lt 21474836629 2026-02-19T13:31:54.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:55.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:55.301 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:55.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836630 -lt 21474836629 2026-02-19T13:31:55.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:55.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705811 2026-02-19T13:31:55.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:55.461 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:55.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705811 2026-02-19T13:31:55.462 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:55.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705811 2026-02-19T13:31:55.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705811' 2026-02-19T13:31:55.464 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705811 2026-02-19T13:31:55.464 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:55.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705811 -lt 38654705811 2026-02-19T13:31:55.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:55.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542288 2026-02-19T13:31:55.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:55.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:55.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542288 2026-02-19T13:31:55.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:55.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542288 2026-02-19T13:31:55.625 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542288' 2026-02-19T13:31:55.625 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542288 2026-02-19T13:31:55.625 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:55.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542289 -lt 60129542288 2026-02-19T13:31:55.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:55.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:55.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:55.931 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:55.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:55.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:55.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:55.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:55.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:56.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:56.091 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:56.092 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:56.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:56.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:56.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:56.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836633 2026-02-19T13:31:56.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836633 2026-02-19T13:31:56.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836633' 2026-02-19T13:31:56.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:56.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:56.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705814 2026-02-19T13:31:56.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705814 2026-02-19T13:31:56.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836633 1-38654705814' 2026-02-19T13:31:56.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:56.237 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:56.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542292 2026-02-19T13:31:56.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542292 2026-02-19T13:31:56.309 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836633 1-38654705814 2-60129542292' 2026-02-19T13:31:56.310 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:56.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836633 2026-02-19T13:31:56.310 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:56.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:56.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836633 2026-02-19T13:31:56.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:56.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836633 2026-02-19T13:31:56.314 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836633 2026-02-19T13:31:56.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836633' 2026-02-19T13:31:56.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:56.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836633 -lt 21474836633 2026-02-19T13:31:56.476 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:56.477 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705814 2026-02-19T13:31:56.477 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:56.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:56.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705814 2026-02-19T13:31:56.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:56.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705814 2026-02-19T13:31:56.480 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705814 2026-02-19T13:31:56.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705814' 2026-02-19T13:31:56.480 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:56.647 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705814 -lt 38654705814 2026-02-19T13:31:56.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:56.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542292 2026-02-19T13:31:56.648 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:56.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:56.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542292 2026-02-19T13:31:56.650 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:56.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542292 2026-02-19T13:31:56.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542292' 2026-02-19T13:31:56.652 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542292 2026-02-19T13:31:56.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:56.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542292 -lt 60129542292 2026-02-19T13:31:56.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:56.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:56.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:56.954 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:56.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:56.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:56.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:56.964 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:56.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:57.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:57.121 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:57.121 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:57.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:57.121 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:57.121 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:57.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836635 2026-02-19T13:31:57.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836635 2026-02-19T13:31:57.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836635' 2026-02-19T13:31:57.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:57.194 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:57.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705816 2026-02-19T13:31:57.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705816 2026-02-19T13:31:57.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836635 1-38654705816' 2026-02-19T13:31:57.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:57.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:57.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542294 2026-02-19T13:31:57.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542294 2026-02-19T13:31:57.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836635 1-38654705816 2-60129542294' 2026-02-19T13:31:57.337 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:57.338 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836635 2026-02-19T13:31:57.338 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:57.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:57.339 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836635 2026-02-19T13:31:57.340 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:57.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836635 2026-02-19T13:31:57.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836635' 2026-02-19T13:31:57.341 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836635 2026-02-19T13:31:57.342 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:57.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836633 -lt 21474836635 2026-02-19T13:31:57.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:31:58.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:31:58.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:58.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836636 -lt 21474836635 2026-02-19T13:31:58.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:58.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705816 2026-02-19T13:31:58.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:58.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:31:58.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705816 2026-02-19T13:31:58.664 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:58.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705816 2026-02-19T13:31:58.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705816' 2026-02-19T13:31:58.666 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705816 2026-02-19T13:31:58.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:31:58.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705817 -lt 38654705816 2026-02-19T13:31:58.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:58.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542294 2026-02-19T13:31:58.825 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:58.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:31:58.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542294 2026-02-19T13:31:58.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:58.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542294 2026-02-19T13:31:58.828 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542294' 2026-02-19T13:31:58.828 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542294 2026-02-19T13:31:58.829 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:31:58.992 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542295 -lt 60129542294 2026-02-19T13:31:58.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:31:58.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:31:58.993 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:31:59.141 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:31:59.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:31:59.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:31:59.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:31:59.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:31:59.153 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:31:59.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:31:59.312 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:31:59.313 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:31:59.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:31:59.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:59.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:31:59.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836638 2026-02-19T13:31:59.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836638 2026-02-19T13:31:59.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836638' 2026-02-19T13:31:59.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:59.383 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:31:59.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705819 2026-02-19T13:31:59.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705819 2026-02-19T13:31:59.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836638 1-38654705819' 2026-02-19T13:31:59.456 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:31:59.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:31:59.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542297 2026-02-19T13:31:59.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542297 2026-02-19T13:31:59.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836638 1-38654705819 2-60129542297' 2026-02-19T13:31:59.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:31:59.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836638 2026-02-19T13:31:59.530 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:31:59.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:31:59.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836638 2026-02-19T13:31:59.532 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:31:59.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836638 2026-02-19T13:31:59.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836638' 2026-02-19T13:31:59.534 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836638 2026-02-19T13:31:59.534 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:31:59.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836636 -lt 21474836638 2026-02-19T13:31:59.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:00.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:00.701 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:00.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836639 -lt 21474836638 2026-02-19T13:32:00.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:00.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705819 2026-02-19T13:32:00.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:00.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:00.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705819 2026-02-19T13:32:00.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:00.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705819 2026-02-19T13:32:00.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705819' 2026-02-19T13:32:00.862 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705819 2026-02-19T13:32:00.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:01.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705820 -lt 38654705819 2026-02-19T13:32:01.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:01.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542297 2026-02-19T13:32:01.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:01.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:01.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542297 2026-02-19T13:32:01.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:01.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542297 2026-02-19T13:32:01.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542297' 2026-02-19T13:32:01.026 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542297 2026-02-19T13:32:01.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:01.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542298 -lt 60129542297 2026-02-19T13:32:01.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:01.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:01.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:01.345 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:01.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:01.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:01.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:01.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:01.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:01.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:01.510 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:01.510 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:01.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:01.510 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:01.510 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:01.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836641 2026-02-19T13:32:01.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836641 2026-02-19T13:32:01.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836641' 2026-02-19T13:32:01.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:01.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:01.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705823 2026-02-19T13:32:01.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705823 2026-02-19T13:32:01.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836641 1-38654705823' 2026-02-19T13:32:01.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:01.654 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:01.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542300 2026-02-19T13:32:01.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542300 2026-02-19T13:32:01.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836641 1-38654705823 2-60129542300' 2026-02-19T13:32:01.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:01.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836641 2026-02-19T13:32:01.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:01.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:01.727 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836641 2026-02-19T13:32:01.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:01.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836641 2026-02-19T13:32:01.729 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836641 2026-02-19T13:32:01.730 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836641' 2026-02-19T13:32:01.730 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:01.891 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836639 -lt 21474836641 2026-02-19T13:32:01.892 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:02.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:02.893 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:03.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836642 -lt 21474836641 2026-02-19T13:32:03.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:03.054 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705823 2026-02-19T13:32:03.054 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:03.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:03.055 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705823 2026-02-19T13:32:03.055 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:03.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705823 2026-02-19T13:32:03.056 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705823 2026-02-19T13:32:03.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705823' 2026-02-19T13:32:03.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:03.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705823 -lt 38654705823 2026-02-19T13:32:03.215 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:03.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542300 2026-02-19T13:32:03.215 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:03.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:03.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542300 2026-02-19T13:32:03.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:03.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542300 2026-02-19T13:32:03.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542300' 2026-02-19T13:32:03.219 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542300 2026-02-19T13:32:03.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:03.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542301 -lt 60129542300 2026-02-19T13:32:03.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:03.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:03.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:03.532 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:03.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:03.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:03.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:03.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:03.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:03.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:03.700 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:03.700 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:03.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:03.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:03.700 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:03.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836644 2026-02-19T13:32:03.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836644 2026-02-19T13:32:03.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836644' 2026-02-19T13:32:03.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:03.774 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:03.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705826 2026-02-19T13:32:03.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705826 2026-02-19T13:32:03.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836644 1-38654705826' 2026-02-19T13:32:03.846 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:03.846 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:03.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542303 2026-02-19T13:32:03.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542303 2026-02-19T13:32:03.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836644 1-38654705826 2-60129542303' 2026-02-19T13:32:03.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:03.921 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836644 2026-02-19T13:32:03.921 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:03.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:03.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836644 2026-02-19T13:32:03.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:03.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836644 2026-02-19T13:32:03.924 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836644 2026-02-19T13:32:03.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836644' 2026-02-19T13:32:03.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:04.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836642 -lt 21474836644 2026-02-19T13:32:04.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:05.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:05.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:05.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836645 -lt 21474836644 2026-02-19T13:32:05.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:05.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705826 2026-02-19T13:32:05.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:05.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:05.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705826 2026-02-19T13:32:05.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:05.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705826 2026-02-19T13:32:05.246 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705826' 2026-02-19T13:32:05.246 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705826 2026-02-19T13:32:05.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:05.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705826 -lt 38654705826 2026-02-19T13:32:05.412 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:05.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542303 2026-02-19T13:32:05.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:05.414 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:05.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542303 2026-02-19T13:32:05.414 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:05.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542303 2026-02-19T13:32:05.416 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542303 2026-02-19T13:32:05.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542303' 2026-02-19T13:32:05.416 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:05.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542304 -lt 60129542303 2026-02-19T13:32:05.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:05.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:05.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:05.733 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:05.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:05.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:05.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:05.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:05.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:05.894 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:05.895 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:05.895 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:05.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:05.895 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:05.895 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:05.965 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836647 2026-02-19T13:32:05.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836647 2026-02-19T13:32:05.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836647' 2026-02-19T13:32:05.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:05.966 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:06.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705829 2026-02-19T13:32:06.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705829 2026-02-19T13:32:06.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836647 1-38654705829' 2026-02-19T13:32:06.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:06.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:06.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542306 2026-02-19T13:32:06.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542306 2026-02-19T13:32:06.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836647 1-38654705829 2-60129542306' 2026-02-19T13:32:06.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:06.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836647 2026-02-19T13:32:06.110 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:06.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:06.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836647 2026-02-19T13:32:06.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:06.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836647 2026-02-19T13:32:06.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836647' 2026-02-19T13:32:06.113 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836647 2026-02-19T13:32:06.114 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:06.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836645 -lt 21474836647 2026-02-19T13:32:06.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:07.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:07.281 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:07.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836648 -lt 21474836647 2026-02-19T13:32:07.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:07.443 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705829 2026-02-19T13:32:07.443 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:07.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:07.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705829 2026-02-19T13:32:07.445 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:07.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705829 2026-02-19T13:32:07.447 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705829 2026-02-19T13:32:07.447 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705829' 2026-02-19T13:32:07.447 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:07.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705829 -lt 38654705829 2026-02-19T13:32:07.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:07.615 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542306 2026-02-19T13:32:07.615 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:07.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:07.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542306 2026-02-19T13:32:07.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:07.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542306 2026-02-19T13:32:07.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542306' 2026-02-19T13:32:07.619 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542306 2026-02-19T13:32:07.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:07.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542307 -lt 60129542306 2026-02-19T13:32:07.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:07.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:07.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:07.926 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:07.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:07.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:07.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:07.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:07.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:08.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:08.097 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:08.098 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:08.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:08.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:08.098 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:08.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836651 2026-02-19T13:32:08.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836651 2026-02-19T13:32:08.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836651' 2026-02-19T13:32:08.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:08.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:08.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705832 2026-02-19T13:32:08.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705832 2026-02-19T13:32:08.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836651 1-38654705832' 2026-02-19T13:32:08.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:08.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:08.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542310 2026-02-19T13:32:08.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542310 2026-02-19T13:32:08.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836651 1-38654705832 2-60129542310' 2026-02-19T13:32:08.312 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:08.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836651 2026-02-19T13:32:08.313 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:08.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:08.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836651 2026-02-19T13:32:08.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:08.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836651 2026-02-19T13:32:08.316 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836651 2026-02-19T13:32:08.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836651' 2026-02-19T13:32:08.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:08.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836651 -lt 21474836651 2026-02-19T13:32:08.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:08.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705832 2026-02-19T13:32:08.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:08.485 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:08.485 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705832 2026-02-19T13:32:08.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:08.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705832 2026-02-19T13:32:08.487 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705832 2026-02-19T13:32:08.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705832' 2026-02-19T13:32:08.487 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:08.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705832 -lt 38654705832 2026-02-19T13:32:08.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:08.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542310 2026-02-19T13:32:08.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:08.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:08.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542310 2026-02-19T13:32:08.644 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:08.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542310 2026-02-19T13:32:08.646 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542310' 2026-02-19T13:32:08.646 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542310 2026-02-19T13:32:08.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:08.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542310 -lt 60129542310 2026-02-19T13:32:08.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:08.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:08.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:08.958 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:08.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:08.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:08.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:08.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:08.970 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:09.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:09.128 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:09.128 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:09.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:09.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:09.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:09.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836653 2026-02-19T13:32:09.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836653 2026-02-19T13:32:09.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836653' 2026-02-19T13:32:09.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:09.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:09.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705834 2026-02-19T13:32:09.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705834 2026-02-19T13:32:09.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836653 1-38654705834' 2026-02-19T13:32:09.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:09.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:09.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542312 2026-02-19T13:32:09.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542312 2026-02-19T13:32:09.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836653 1-38654705834 2-60129542312' 2026-02-19T13:32:09.348 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:09.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836653 2026-02-19T13:32:09.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:09.350 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:09.351 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836653 2026-02-19T13:32:09.351 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:09.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836653 2026-02-19T13:32:09.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836653' 2026-02-19T13:32:09.353 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836653 2026-02-19T13:32:09.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:09.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836651 -lt 21474836653 2026-02-19T13:32:09.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:10.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:10.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:10.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836654 -lt 21474836653 2026-02-19T13:32:10.682 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:10.682 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705834 2026-02-19T13:32:10.682 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:10.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:10.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705834 2026-02-19T13:32:10.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:10.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705834 2026-02-19T13:32:10.686 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705834' 2026-02-19T13:32:10.686 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705834 2026-02-19T13:32:10.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:10.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705835 -lt 38654705834 2026-02-19T13:32:10.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:10.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542312 2026-02-19T13:32:10.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:10.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:10.856 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542312 2026-02-19T13:32:10.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:10.858 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542312 2026-02-19T13:32:10.858 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542312 2026-02-19T13:32:10.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542312' 2026-02-19T13:32:10.859 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:11.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542313 -lt 60129542312 2026-02-19T13:32:11.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:11.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:11.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:11.169 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:11.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:11.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:11.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:11.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:11.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:11.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:11.338 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:11.338 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:11.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:11.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:11.338 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:11.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836656 2026-02-19T13:32:11.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836656 2026-02-19T13:32:11.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836656' 2026-02-19T13:32:11.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:11.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:11.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705837 2026-02-19T13:32:11.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705837 2026-02-19T13:32:11.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836656 1-38654705837' 2026-02-19T13:32:11.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:11.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:11.553 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542315 2026-02-19T13:32:11.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542315 2026-02-19T13:32:11.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836656 1-38654705837 2-60129542315' 2026-02-19T13:32:11.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:11.554 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836656 2026-02-19T13:32:11.554 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:11.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:11.556 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836656 2026-02-19T13:32:11.556 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:11.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836656 2026-02-19T13:32:11.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836656' 2026-02-19T13:32:11.558 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836656 2026-02-19T13:32:11.558 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:11.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836654 -lt 21474836656 2026-02-19T13:32:11.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:12.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:12.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:12.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836657 -lt 21474836656 2026-02-19T13:32:12.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:12.885 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705837 2026-02-19T13:32:12.885 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:12.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:12.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705837 2026-02-19T13:32:12.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:12.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705837 2026-02-19T13:32:12.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705837' 2026-02-19T13:32:12.889 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705837 2026-02-19T13:32:12.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:13.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705838 -lt 38654705837 2026-02-19T13:32:13.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:13.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542315 2026-02-19T13:32:13.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:13.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:13.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542315 2026-02-19T13:32:13.062 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:13.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542315 2026-02-19T13:32:13.063 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542315 2026-02-19T13:32:13.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542315' 2026-02-19T13:32:13.064 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:13.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542316 -lt 60129542315 2026-02-19T13:32:13.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:13.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:13.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:13.378 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:13.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:13.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:13.388 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:13.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:13.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:13.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:13.548 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:13.548 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:13.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:13.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:13.549 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:13.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836659 2026-02-19T13:32:13.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836659 2026-02-19T13:32:13.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836659' 2026-02-19T13:32:13.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:13.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:13.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705841 2026-02-19T13:32:13.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705841 2026-02-19T13:32:13.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836659 1-38654705841' 2026-02-19T13:32:13.695 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:13.695 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:13.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542318 2026-02-19T13:32:13.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542318 2026-02-19T13:32:13.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836659 1-38654705841 2-60129542318' 2026-02-19T13:32:13.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:13.765 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836659 2026-02-19T13:32:13.766 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:13.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:13.767 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836659 2026-02-19T13:32:13.768 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:13.769 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836659 2026-02-19T13:32:13.769 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836659 2026-02-19T13:32:13.770 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836659' 2026-02-19T13:32:13.770 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:13.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836657 -lt 21474836659 2026-02-19T13:32:13.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:14.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:14.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:15.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836660 -lt 21474836659 2026-02-19T13:32:15.096 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:15.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705841 2026-02-19T13:32:15.097 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:15.098 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:15.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705841 2026-02-19T13:32:15.099 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:15.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705841 2026-02-19T13:32:15.101 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705841 2026-02-19T13:32:15.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705841' 2026-02-19T13:32:15.101 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:15.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705841 -lt 38654705841 2026-02-19T13:32:15.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:15.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542318 2026-02-19T13:32:15.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:15.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:15.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542318 2026-02-19T13:32:15.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:15.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542318 2026-02-19T13:32:15.277 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542318 2026-02-19T13:32:15.278 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542318' 2026-02-19T13:32:15.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:15.444 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542319 -lt 60129542318 2026-02-19T13:32:15.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:15.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:15.445 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:15.589 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:15.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:15.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:15.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:15.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:15.601 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:15.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:15.762 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:15.762 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:15.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:15.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:15.762 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:15.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836662 2026-02-19T13:32:15.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836662 2026-02-19T13:32:15.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836662' 2026-02-19T13:32:15.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:15.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:15.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705844 2026-02-19T13:32:15.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705844 2026-02-19T13:32:15.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836662 1-38654705844' 2026-02-19T13:32:15.906 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:15.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:15.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542321 2026-02-19T13:32:15.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542321 2026-02-19T13:32:15.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836662 1-38654705844 2-60129542321' 2026-02-19T13:32:15.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:15.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836662 2026-02-19T13:32:15.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:15.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:15.978 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836662 2026-02-19T13:32:15.978 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:15.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836662 2026-02-19T13:32:15.980 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836662 2026-02-19T13:32:15.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836662' 2026-02-19T13:32:15.980 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:16.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836660 -lt 21474836662 2026-02-19T13:32:16.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:17.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:17.140 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:17.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836663 -lt 21474836662 2026-02-19T13:32:17.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:17.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705844 2026-02-19T13:32:17.295 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:17.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:17.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705844 2026-02-19T13:32:17.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:17.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705844 2026-02-19T13:32:17.299 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705844 2026-02-19T13:32:17.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705844' 2026-02-19T13:32:17.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:17.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705844 -lt 38654705844 2026-02-19T13:32:17.464 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:17.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542321 2026-02-19T13:32:17.465 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:17.466 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:17.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542321 2026-02-19T13:32:17.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:17.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542321 2026-02-19T13:32:17.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542321' 2026-02-19T13:32:17.469 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542321 2026-02-19T13:32:17.469 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:17.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542322 -lt 60129542321 2026-02-19T13:32:17.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:17.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:17.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:17.781 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:17.790 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:17.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:17.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:17.791 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:17.791 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:17.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:17.948 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:17.948 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:17.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:17.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:17.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:18.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836665 2026-02-19T13:32:18.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836665 2026-02-19T13:32:18.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836665' 2026-02-19T13:32:18.024 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:18.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:18.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705847 2026-02-19T13:32:18.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705847 2026-02-19T13:32:18.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836665 1-38654705847' 2026-02-19T13:32:18.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:18.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:18.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542324 2026-02-19T13:32:18.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542324 2026-02-19T13:32:18.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836665 1-38654705847 2-60129542324' 2026-02-19T13:32:18.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:18.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836665 2026-02-19T13:32:18.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:18.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:18.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836665 2026-02-19T13:32:18.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:18.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836665 2026-02-19T13:32:18.169 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836665 2026-02-19T13:32:18.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836665' 2026-02-19T13:32:18.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:18.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836663 -lt 21474836665 2026-02-19T13:32:18.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:19.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:19.336 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:19.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836666 -lt 21474836665 2026-02-19T13:32:19.499 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:19.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705847 2026-02-19T13:32:19.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:19.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:19.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705847 2026-02-19T13:32:19.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:19.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705847 2026-02-19T13:32:19.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705847' 2026-02-19T13:32:19.504 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705847 2026-02-19T13:32:19.504 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:19.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705847 -lt 38654705847 2026-02-19T13:32:19.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:19.664 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542324 2026-02-19T13:32:19.664 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:19.665 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:19.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542324 2026-02-19T13:32:19.666 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:19.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542324 2026-02-19T13:32:19.667 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542324' 2026-02-19T13:32:19.668 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542324 2026-02-19T13:32:19.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:19.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542325 -lt 60129542324 2026-02-19T13:32:19.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:19.825 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:19.826 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:19.973 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:19.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:19.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:19.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:19.984 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:19.985 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:20.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:20.142 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:20.142 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:20.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:20.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:20.142 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:20.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836669 2026-02-19T13:32:20.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836669 2026-02-19T13:32:20.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836669' 2026-02-19T13:32:20.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:20.214 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:20.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705850 2026-02-19T13:32:20.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705850 2026-02-19T13:32:20.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836669 1-38654705850' 2026-02-19T13:32:20.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:20.288 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:20.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542328 2026-02-19T13:32:20.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542328 2026-02-19T13:32:20.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836669 1-38654705850 2-60129542328' 2026-02-19T13:32:20.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:20.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836669 2026-02-19T13:32:20.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:20.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:20.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836669 2026-02-19T13:32:20.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:20.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836669 2026-02-19T13:32:20.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836669' 2026-02-19T13:32:20.362 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836669 2026-02-19T13:32:20.362 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:20.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836669 -lt 21474836669 2026-02-19T13:32:20.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:20.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705850 2026-02-19T13:32:20.527 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:20.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:20.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705850 2026-02-19T13:32:20.529 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:20.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705850 2026-02-19T13:32:20.531 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705850 2026-02-19T13:32:20.531 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705850' 2026-02-19T13:32:20.531 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:20.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705850 -lt 38654705850 2026-02-19T13:32:20.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:20.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542328 2026-02-19T13:32:20.685 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:20.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:20.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542328 2026-02-19T13:32:20.688 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:20.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542328 2026-02-19T13:32:20.689 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542328 2026-02-19T13:32:20.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542328' 2026-02-19T13:32:20.689 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:20.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542327 -lt 60129542328 2026-02-19T13:32:20.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:21.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:21.856 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:22.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542327 -lt 60129542328 2026-02-19T13:32:22.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:23.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-19T13:32:23.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:23.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542330 -lt 60129542328 2026-02-19T13:32:23.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:23.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:23.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:23.323 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:23.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:23.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:23.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:23.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:23.335 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:23.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:23.493 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:23.493 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:23.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:23.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:23.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:23.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836673 2026-02-19T13:32:23.568 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836673 2026-02-19T13:32:23.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836673' 2026-02-19T13:32:23.569 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:23.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:23.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705855 2026-02-19T13:32:23.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705855 2026-02-19T13:32:23.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836673 1-38654705855' 2026-02-19T13:32:23.643 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:23.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:23.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542332 2026-02-19T13:32:23.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542332 2026-02-19T13:32:23.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836673 1-38654705855 2-60129542332' 2026-02-19T13:32:23.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:23.716 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836673 2026-02-19T13:32:23.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:23.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:23.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836673 2026-02-19T13:32:23.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:23.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836673 2026-02-19T13:32:23.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836673' 2026-02-19T13:32:23.720 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836673 2026-02-19T13:32:23.721 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:23.883 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836671 -lt 21474836673 2026-02-19T13:32:23.884 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:24.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:24.885 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:25.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836674 -lt 21474836673 2026-02-19T13:32:25.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:25.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705855 2026-02-19T13:32:25.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:25.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:25.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705855 2026-02-19T13:32:25.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:25.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705855 2026-02-19T13:32:25.048 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705855 2026-02-19T13:32:25.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705855' 2026-02-19T13:32:25.048 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:25.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705855 -lt 38654705855 2026-02-19T13:32:25.216 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:25.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542332 2026-02-19T13:32:25.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:25.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:25.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542332 2026-02-19T13:32:25.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:25.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542332 2026-02-19T13:32:25.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542332' 2026-02-19T13:32:25.221 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542332 2026-02-19T13:32:25.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:25.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542333 -lt 60129542332 2026-02-19T13:32:25.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:25.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:25.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:25.534 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:25.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:25.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:25.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:25.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:25.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:25.699 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:25.699 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:25.699 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:25.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:25.700 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:25.700 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:25.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836676 2026-02-19T13:32:25.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836676 2026-02-19T13:32:25.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836676' 2026-02-19T13:32:25.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:25.774 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:25.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705858 2026-02-19T13:32:25.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705858 2026-02-19T13:32:25.844 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836676 1-38654705858' 2026-02-19T13:32:25.845 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:25.845 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:25.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542335 2026-02-19T13:32:25.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542335 2026-02-19T13:32:25.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836676 1-38654705858 2-60129542335' 2026-02-19T13:32:25.917 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:25.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836676 2026-02-19T13:32:25.917 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:25.918 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:25.919 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836676 2026-02-19T13:32:25.919 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:25.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836676 2026-02-19T13:32:25.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836676' 2026-02-19T13:32:25.921 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836676 2026-02-19T13:32:25.921 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:26.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836674 -lt 21474836676 2026-02-19T13:32:26.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:27.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:27.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:27.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836677 -lt 21474836676 2026-02-19T13:32:27.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:27.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705858 2026-02-19T13:32:27.246 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:27.247 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:27.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705858 2026-02-19T13:32:27.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:27.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705858 2026-02-19T13:32:27.250 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705858' 2026-02-19T13:32:27.250 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705858 2026-02-19T13:32:27.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:27.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705858 -lt 38654705858 2026-02-19T13:32:27.421 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:27.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542335 2026-02-19T13:32:27.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:27.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:27.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542335 2026-02-19T13:32:27.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:27.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542335 2026-02-19T13:32:27.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542335' 2026-02-19T13:32:27.426 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542335 2026-02-19T13:32:27.426 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:27.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542336 -lt 60129542335 2026-02-19T13:32:27.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:27.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:27.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:27.740 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:27.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:27.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:27.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:27.750 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:27.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:27.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:27.905 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:27.905 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:27.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:27.905 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:27.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:27.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836679 2026-02-19T13:32:27.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836679 2026-02-19T13:32:27.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836679' 2026-02-19T13:32:27.977 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:27.977 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:28.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705861 2026-02-19T13:32:28.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705861 2026-02-19T13:32:28.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836679 1-38654705861' 2026-02-19T13:32:28.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:28.050 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:28.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542338 2026-02-19T13:32:28.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542338 2026-02-19T13:32:28.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836679 1-38654705861 2-60129542338' 2026-02-19T13:32:28.120 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:28.121 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836679 2026-02-19T13:32:28.121 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:28.122 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:28.123 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836679 2026-02-19T13:32:28.123 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:28.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836679 2026-02-19T13:32:28.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836679' 2026-02-19T13:32:28.124 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836679 2026-02-19T13:32:28.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:28.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836677 -lt 21474836679 2026-02-19T13:32:28.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:29.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:29.291 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:29.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836680 -lt 21474836679 2026-02-19T13:32:29.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:29.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705861 2026-02-19T13:32:29.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:29.452 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:29.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705861 2026-02-19T13:32:29.453 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:29.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705861 2026-02-19T13:32:29.455 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705861 2026-02-19T13:32:29.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705861' 2026-02-19T13:32:29.455 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:29.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705861 -lt 38654705861 2026-02-19T13:32:29.616 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:29.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542338 2026-02-19T13:32:29.617 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:29.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:29.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542338 2026-02-19T13:32:29.619 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:29.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542338 2026-02-19T13:32:29.621 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542338 2026-02-19T13:32:29.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542338' 2026-02-19T13:32:29.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:29.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542339 -lt 60129542338 2026-02-19T13:32:29.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:29.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:29.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:29.933 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:29.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:29.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:29.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:29.943 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:29.943 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:30.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:30.099 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:30.099 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:30.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:30.099 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:30.100 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:30.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836683 2026-02-19T13:32:30.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836683 2026-02-19T13:32:30.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836683' 2026-02-19T13:32:30.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:30.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:30.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705864 2026-02-19T13:32:30.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705864 2026-02-19T13:32:30.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836683 1-38654705864' 2026-02-19T13:32:30.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:30.244 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:30.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542342 2026-02-19T13:32:30.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542342 2026-02-19T13:32:30.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836683 1-38654705864 2-60129542342' 2026-02-19T13:32:30.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:30.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836683 2026-02-19T13:32:30.321 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:30.321 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:30.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836683 2026-02-19T13:32:30.322 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:30.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836683 2026-02-19T13:32:30.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836683' 2026-02-19T13:32:30.324 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836683 2026-02-19T13:32:30.324 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:30.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836683 -lt 21474836683 2026-02-19T13:32:30.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:30.481 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705864 2026-02-19T13:32:30.481 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:30.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:30.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705864 2026-02-19T13:32:30.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:30.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705864 2026-02-19T13:32:30.484 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705864' 2026-02-19T13:32:30.485 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705864 2026-02-19T13:32:30.485 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:30.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705864 -lt 38654705864 2026-02-19T13:32:30.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:30.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542342 2026-02-19T13:32:30.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:30.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:30.657 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542342 2026-02-19T13:32:30.657 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:30.658 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542342 2026-02-19T13:32:30.659 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542342 2026-02-19T13:32:30.659 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542342' 2026-02-19T13:32:30.659 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:30.820 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542342 -lt 60129542342 2026-02-19T13:32:30.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:30.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:30.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:30.969 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:30.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:30.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:30.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:30.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:30.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:31.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:31.139 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:31.139 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:31.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:31.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:31.139 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:31.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836685 2026-02-19T13:32:31.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836685 2026-02-19T13:32:31.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836685' 2026-02-19T13:32:31.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:31.210 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:31.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705866 2026-02-19T13:32:31.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705866 2026-02-19T13:32:31.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836685 1-38654705866' 2026-02-19T13:32:31.281 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:31.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:31.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542344 2026-02-19T13:32:31.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542344 2026-02-19T13:32:31.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836685 1-38654705866 2-60129542344' 2026-02-19T13:32:31.352 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:31.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836685 2026-02-19T13:32:31.353 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:31.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:31.355 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836685 2026-02-19T13:32:31.355 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:31.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836685 2026-02-19T13:32:31.356 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836685 2026-02-19T13:32:31.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836685' 2026-02-19T13:32:31.357 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:31.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836683 -lt 21474836685 2026-02-19T13:32:31.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:32.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:32.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:32.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836686 -lt 21474836685 2026-02-19T13:32:32.681 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:32.682 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705866 2026-02-19T13:32:32.682 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:32.683 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:32.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705866 2026-02-19T13:32:32.684 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:32.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705866 2026-02-19T13:32:32.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705866' 2026-02-19T13:32:32.686 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705866 2026-02-19T13:32:32.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:32.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705867 -lt 38654705866 2026-02-19T13:32:32.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:32.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542344 2026-02-19T13:32:32.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:32.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:32.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542344 2026-02-19T13:32:32.852 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:32.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542344 2026-02-19T13:32:32.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542344' 2026-02-19T13:32:32.854 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542344 2026-02-19T13:32:32.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:33.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542345 -lt 60129542344 2026-02-19T13:32:33.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:33.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:33.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:33.167 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:33.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:33.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:33.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:33.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:33.179 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:33.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:33.338 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:33.338 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:33.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:33.338 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:33.338 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:33.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836688 2026-02-19T13:32:33.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836688 2026-02-19T13:32:33.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836688' 2026-02-19T13:32:33.411 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:33.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:33.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705869 2026-02-19T13:32:33.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705869 2026-02-19T13:32:33.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836688 1-38654705869' 2026-02-19T13:32:33.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:33.481 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:33.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542347 2026-02-19T13:32:33.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542347 2026-02-19T13:32:33.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836688 1-38654705869 2-60129542347' 2026-02-19T13:32:33.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:33.553 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836688 2026-02-19T13:32:33.553 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:33.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:33.555 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836688 2026-02-19T13:32:33.555 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:33.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836688 2026-02-19T13:32:33.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836688' 2026-02-19T13:32:33.557 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836688 2026-02-19T13:32:33.557 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:33.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836686 -lt 21474836688 2026-02-19T13:32:33.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:34.728 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:34.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:34.888 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836689 -lt 21474836688 2026-02-19T13:32:34.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:34.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705869 2026-02-19T13:32:34.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:34.890 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:34.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705869 2026-02-19T13:32:34.891 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:34.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705869 2026-02-19T13:32:34.893 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705869 2026-02-19T13:32:34.893 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705869' 2026-02-19T13:32:34.893 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:35.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705870 -lt 38654705869 2026-02-19T13:32:35.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:35.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542347 2026-02-19T13:32:35.057 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:35.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:35.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542347 2026-02-19T13:32:35.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:35.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542347 2026-02-19T13:32:35.061 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542347 2026-02-19T13:32:35.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542347' 2026-02-19T13:32:35.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:35.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542348 -lt 60129542347 2026-02-19T13:32:35.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:35.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:35.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:35.375 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:35.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:35.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:35.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:35.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:35.385 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:35.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:35.544 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:35.545 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:35.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:35.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:35.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:35.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836691 2026-02-19T13:32:35.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836691 2026-02-19T13:32:35.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836691' 2026-02-19T13:32:35.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:35.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:35.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705873 2026-02-19T13:32:35.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705873 2026-02-19T13:32:35.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836691 1-38654705873' 2026-02-19T13:32:35.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:35.691 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:35.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542350 2026-02-19T13:32:35.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542350 2026-02-19T13:32:35.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836691 1-38654705873 2-60129542350' 2026-02-19T13:32:35.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:35.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836691 2026-02-19T13:32:35.761 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:35.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:35.763 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836691 2026-02-19T13:32:35.763 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:35.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836691 2026-02-19T13:32:35.765 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836691' 2026-02-19T13:32:35.765 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836691 2026-02-19T13:32:35.765 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:35.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836689 -lt 21474836691 2026-02-19T13:32:35.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:36.934 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:36.935 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:37.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836692 -lt 21474836691 2026-02-19T13:32:37.091 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:37.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705873 2026-02-19T13:32:37.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:37.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:37.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705873 2026-02-19T13:32:37.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:37.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705873 2026-02-19T13:32:37.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705873' 2026-02-19T13:32:37.095 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705873 2026-02-19T13:32:37.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:37.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705873 -lt 38654705873 2026-02-19T13:32:37.252 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:37.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542350 2026-02-19T13:32:37.253 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:37.254 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:37.255 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542350 2026-02-19T13:32:37.255 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:37.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542350 2026-02-19T13:32:37.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542350' 2026-02-19T13:32:37.257 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542350 2026-02-19T13:32:37.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:37.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542351 -lt 60129542350 2026-02-19T13:32:37.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:37.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:37.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:37.569 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:37.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:37.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:37.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:37.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:37.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:37.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:37.736 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:37.737 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:37.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:37.737 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:37.737 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:37.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836694 2026-02-19T13:32:37.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836694 2026-02-19T13:32:37.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836694' 2026-02-19T13:32:37.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:37.809 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:37.880 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705876 2026-02-19T13:32:37.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705876 2026-02-19T13:32:37.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836694 1-38654705876' 2026-02-19T13:32:37.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:37.881 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:37.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542353 2026-02-19T13:32:37.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542353 2026-02-19T13:32:37.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836694 1-38654705876 2-60129542353' 2026-02-19T13:32:37.955 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:37.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836694 2026-02-19T13:32:37.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:37.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:37.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836694 2026-02-19T13:32:37.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:37.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836694 2026-02-19T13:32:37.959 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836694' 2026-02-19T13:32:37.960 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836694 2026-02-19T13:32:37.960 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:38.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836692 -lt 21474836694 2026-02-19T13:32:38.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:39.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:39.118 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:39.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836695 -lt 21474836694 2026-02-19T13:32:39.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:39.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705876 2026-02-19T13:32:39.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:39.277 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:39.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705876 2026-02-19T13:32:39.278 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:39.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705876 2026-02-19T13:32:39.280 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705876 2026-02-19T13:32:39.280 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705876' 2026-02-19T13:32:39.280 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:39.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705876 -lt 38654705876 2026-02-19T13:32:39.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:39.447 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542353 2026-02-19T13:32:39.447 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:39.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:39.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542353 2026-02-19T13:32:39.449 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:39.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542353 2026-02-19T13:32:39.451 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542353 2026-02-19T13:32:39.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542353' 2026-02-19T13:32:39.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:39.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542354 -lt 60129542353 2026-02-19T13:32:39.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:39.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:39.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:39.768 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:39.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:39.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:39.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:39.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:39.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:39.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:39.937 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:39.938 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:39.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:39.938 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:39.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:40.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836697 2026-02-19T13:32:40.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836697 2026-02-19T13:32:40.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836697' 2026-02-19T13:32:40.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:40.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:40.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705879 2026-02-19T13:32:40.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705879 2026-02-19T13:32:40.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836697 1-38654705879' 2026-02-19T13:32:40.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:40.094 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:40.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542356 2026-02-19T13:32:40.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542356 2026-02-19T13:32:40.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836697 1-38654705879 2-60129542356' 2026-02-19T13:32:40.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:40.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836697 2026-02-19T13:32:40.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:40.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:40.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836697 2026-02-19T13:32:40.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:40.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836697 2026-02-19T13:32:40.169 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836697 2026-02-19T13:32:40.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836697' 2026-02-19T13:32:40.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:40.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836695 -lt 21474836697 2026-02-19T13:32:40.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:32:41.336 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:32:41.337 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:41.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836698 -lt 21474836697 2026-02-19T13:32:41.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:41.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705879 2026-02-19T13:32:41.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:41.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:41.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705879 2026-02-19T13:32:41.501 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:41.502 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705879 2026-02-19T13:32:41.502 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705879 2026-02-19T13:32:41.503 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705879' 2026-02-19T13:32:41.503 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:41.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705879 -lt 38654705879 2026-02-19T13:32:41.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:41.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542356 2026-02-19T13:32:41.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:41.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:41.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542356 2026-02-19T13:32:41.669 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:41.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542356 2026-02-19T13:32:41.671 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542356 2026-02-19T13:32:41.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542356' 2026-02-19T13:32:41.671 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:41.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542357 -lt 60129542356 2026-02-19T13:32:41.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:41.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:41.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:41.976 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:41.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: continue 2026-02-19T13:32:41.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: for i in $(seq 0 200) 2026-02-19T13:32:41.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:372: _scrub_abort: flush_pg_stats 2026-02-19T13:32:41.985 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:32:41.985 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:32:42.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:32:42.138 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:32:42.138 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:32:42.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:32:42.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:42.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:32:42.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836701 2026-02-19T13:32:42.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836701 2026-02-19T13:32:42.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836701' 2026-02-19T13:32:42.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:42.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:32:42.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705882 2026-02-19T13:32:42.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705882 2026-02-19T13:32:42.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836701 1-38654705882' 2026-02-19T13:32:42.279 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:32:42.279 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:32:42.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542360 2026-02-19T13:32:42.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542360 2026-02-19T13:32:42.353 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836701 1-38654705882 2-60129542360' 2026-02-19T13:32:42.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:42.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836701 2026-02-19T13:32:42.354 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:42.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:32:42.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836701 2026-02-19T13:32:42.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:42.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836701 2026-02-19T13:32:42.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836701' 2026-02-19T13:32:42.358 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836701 2026-02-19T13:32:42.358 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:32:42.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836701 -lt 21474836701 2026-02-19T13:32:42.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:42.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705882 2026-02-19T13:32:42.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:42.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:32:42.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705882 2026-02-19T13:32:42.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:42.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705882 2026-02-19T13:32:42.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705882' 2026-02-19T13:32:42.527 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705882 2026-02-19T13:32:42.528 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:32:42.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705882 -lt 38654705882 2026-02-19T13:32:42.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:32:42.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542360 2026-02-19T13:32:42.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:32:42.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:32:42.688 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542360 2026-02-19T13:32:42.688 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:32:42.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542360 2026-02-19T13:32:42.689 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542360 2026-02-19T13:32:42.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542360' 2026-02-19T13:32:42.690 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:32:42.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542360 -lt 60129542360 2026-02-19T13:32:42.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: ceph pg dump pgs 2026-02-19T13:32:42.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep '^1.0' 2026-02-19T13:32:42.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:373: _scrub_abort: grep -q scrubbing 2026-02-19T13:32:42.998 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:32:43.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: break 2026-02-19T13:32:43.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: set +o pipefail 2026-02-19T13:32:43.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:382: _scrub_abort: sleep 5 2026-02-19T13:32:48.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:384: _scrub_abort: grep 'noscrub set, aborting' td/osd-scrub-test/osd.1.log 2026-02-19T13:32:48.018 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:30:46.137+0000 7f42b7fb9640 10 osd.1 pg_epoch: 18 pg[1.0( v 17'1000 (0'0,17'1000] local-lis/les=15/16 n=1000 ec=15/15 lis/c=15/15 les/c/f=16/16/0 sis=15) [1,0,2] r=0 lpr=15 crt=17'1000 lcod 17'999 mlcod 17'999 active+clean+scrubbing(periodic-regular)] scrubber: noscrub set, aborting 2026-02-19T13:32:48.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:390: _scrub_abort: get_last_scrub_stamp 1.0 2026-02-19T13:32:48.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:32:48.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:32:48.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:32:48.019 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:32:48.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:390: _scrub_abort: local last_scrub=2026-02-17T13:30:37.100168+0000 2026-02-19T13:32:48.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:391: _scrub_abort: ceph config set osd osd_scrub_sleep 0.1 2026-02-19T13:32:48.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:393: _scrub_abort: ceph osd unset noscrub 2026-02-19T13:32:48.526 INFO:tasks.workunit.client.0.trial144.stderr:noscrub is unset 2026-02-19T13:32:48.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:394: _scrub_abort: '[' scrub = deep-scrub ']' 2026-02-19T13:32:48.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:398: _scrub_abort: TIMEOUT=500 2026-02-19T13:32:48.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:399: _scrub_abort: wait_for_scrub 1.0 2026-02-17T13:30:37.100168+0000 2026-02-19T13:32:48.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T13:32:48.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-17T13:30:37.100168+0000 2026-02-19T13:32:48.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:32:48.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:32:48.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:32:48.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:32:48.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:32:48.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:32:48.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:32:48.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:32:48.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:32:48.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:32:49.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:32:49.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:32:49.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:32:49.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:32:49.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:32:49.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:32:49.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:32:49.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:32:49.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:32:50.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:32:50.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:32:50.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:32:50.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:32:50.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:32:50.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:32:50.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:32:51.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:32:51.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:32:52.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:32:52.011 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:32:52.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:32:52.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:32:52.011 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:32:52.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:32:52.012 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:32:52.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:32:52.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:32:53.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:32:53.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:32:53.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:32:53.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:32:53.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:32:53.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:32:53.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:32:53.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:32:53.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:32:54.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:32:54.325 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:32:54.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:32:54.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:32:54.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:32:54.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:32:54.326 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:32:54.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:32:54.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:32:55.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:32:55.482 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:32:55.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:32:55.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:32:55.483 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:32:55.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:32:55.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:32:55.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:32:55.640 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:32:56.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:32:56.642 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:32:56.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:32:56.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:32:56.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:32:56.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:32:56.643 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:32:56.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:32:56.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:32:57.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:32:57.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:32:57.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:32:57.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:32:57.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:32:57.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:32:57.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:32:57.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:32:57.956 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:32:58.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:32:58.957 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:32:58.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:32:58.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:32:58.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:32:58.958 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:32:58.959 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:32:59.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:32:59.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:00.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:00.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:00.115 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:00.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:00.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:00.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:00.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:00.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:00.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:01.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:01.272 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:01.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:01.272 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:01.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:01.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:01.273 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:01.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:01.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:02.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:02.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:02.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:02.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:02.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:02.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:02.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:02.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:02.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:03.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:03.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:03.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:03.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:03.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:03.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:03.587 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:03.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:03.738 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:04.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:04.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:04.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:04.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:04.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:04.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:04.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:04.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:04.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:05.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:05.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:05.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:05.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:05.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:05.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:05.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:06.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:06.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:07.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:07.073 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:07.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:07.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:07.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:07.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:07.073 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:07.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:07.218 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:08.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:08.220 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:08.220 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:08.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:08.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:08.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:08.221 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:08.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:08.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:09.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:09.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:09.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:09.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:09.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:09.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:09.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:09.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:09.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:10.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:10.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:10.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:10.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:10.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:10.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:10.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:10.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:10.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:11.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:11.702 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:11.702 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:11.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:11.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:11.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:11.703 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:11.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:11.861 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:12.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:12.862 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:12.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:12.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:12.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:12.863 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:12.864 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:13.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:13.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:14.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:14.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:14.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:14.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:14.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:14.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:14.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:14.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:14.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:15.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:15.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:15.183 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:15.183 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:15.183 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:15.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:15.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:15.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:15.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:16.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:16.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:16.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:16.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:16.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:16.344 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:16.345 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:16.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:16.500 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:17.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:17.501 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:17.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:17.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:17.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:17.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:17.502 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:17.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:17.660 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:18.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:18.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:18.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:18.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:18.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:18.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:18.663 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:18.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:18.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:19.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:19.823 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:19.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:19.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:19.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:19.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:19.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:19.979 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:19.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:20.980 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:20.981 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:20.981 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:20.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:20.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:20.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:20.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:21.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:21.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:22.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:22.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:22.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:22.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:22.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:22.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:22.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:22.294 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:22.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:23.295 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:23.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:23.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:23.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:23.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:23.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:23.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:23.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:23.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:24.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:24.459 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:24.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:24.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:24.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:24.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:24.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:24.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:24.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:25.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:25.622 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:25.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:25.623 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:25.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:25.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:25.624 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:25.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:25.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:26.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:26.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:26.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:26.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:26.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:26.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:26.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:26.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:26.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:27.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:27.942 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:27.943 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:27.943 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:27.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:27.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:27.944 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:28.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:28.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:29.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:29.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:29.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:29.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:29.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:29.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:29.103 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:29.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:29.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:30.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:30.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:30.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:30.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:30.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:30.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:30.265 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:30.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:30.419 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:31.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:31.420 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:31.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:31.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:31.421 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:31.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:31.422 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:31.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:31.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:32.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:32.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:32.583 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:32.583 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:32.583 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:32.583 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:32.584 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:32.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:32.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:33.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:33.740 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:33.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:33.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:33.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:33.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:33.742 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:33.896 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:33.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:34.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:34.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:34.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:34.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:34.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:34.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:34.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:35.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:35.056 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:36.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:36.057 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:36.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:36.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:36.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:36.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:36.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:36.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:36.219 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:37.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:37.221 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:37.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:37.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:37.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:37.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:37.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:37.377 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:37.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:38.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:38.379 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:38.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:38.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:38.380 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:38.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:38.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:38.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:38.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:39.543 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:39.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:39.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:39.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:39.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:39.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:39.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:39.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:39.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:40.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:40.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:40.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:40.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:40.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:40.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:40.699 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:40.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:40.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:41.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:41.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:41.858 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:41.858 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:41.858 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:41.858 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:41.858 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:42.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:42.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:43.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:43.015 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:43.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:43.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:43.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:43.016 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:43.017 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:43.172 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:43.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:44.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:44.174 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:44.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:44.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:44.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:44.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:44.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:44.327 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:44.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:45.328 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:45.329 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:45.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:45.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:45.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:45.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:45.330 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:45.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:45.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:46.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:46.487 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:46.488 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:46.488 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:46.488 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:46.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:46.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:46.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:46.644 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:47.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:47.645 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:47.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:47.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:47.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:47.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:47.646 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:47.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:47.799 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:48.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:48.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:48.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:48.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:48.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:48.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:48.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:48.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:48.960 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:49.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:49.962 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:49.962 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:49.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:49.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:49.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:49.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:50.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:50.117 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:51.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:51.118 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:51.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:51.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:51.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:51.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:51.119 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:51.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:51.273 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:52.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:52.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:52.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:52.275 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:52.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:52.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:52.276 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:52.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:52.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:53.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:53.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:53.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:53.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:53.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:53.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:53.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:53.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:53.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:54.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:54.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:54.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:54.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:54.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:54.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:54.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:54.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:54.744 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:55.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:55.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:55.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:55.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:55.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:55.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:55.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:55.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:55.900 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:56.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:56.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:56.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:56.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:56.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:56.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:56.902 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:57.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:57.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:58.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:58.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:58.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:58.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:58.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:58.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:58.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:58.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:58.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:33:59.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:33:59.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:33:59.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:33:59.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:33:59.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:33:59.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:33:59.225 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:33:59.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:33:59.382 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:34:00.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:34:00.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:34:00.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:34:00.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:00.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:00.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:00.384 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:00.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:34:00.540 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:34:01.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:34:01.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:34:01.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:34:01.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:01.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:01.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:01.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:01.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:34:01.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:34:02.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:34:02.697 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:34:02.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:34:02.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:02.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:02.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:02.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:02.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:34:02.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:34:03.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:34:03.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:34:03.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:34:03.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:03.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:03.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:03.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:04.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:34:04.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:34:05.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:34:05.008 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:34:05.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:34:05.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:05.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:05.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:05.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:05.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:34:05.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:34:06.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:34:06.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:34:06.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:34:06.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:06.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:06.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:06.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:06.322 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:34:06.323 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:34:07.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:34:07.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:34:07.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:34:07.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:07.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:07.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:07.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:07.478 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:34:07.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:34:08.479 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:34:08.480 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:34:08.480 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:34:08.481 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:08.481 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:08.481 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:08.481 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:08.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:34:08.637 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:34:09.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:34:09.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:34:09.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:34:09.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:09.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:09.639 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:09.640 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:09.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:34:09.794 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:34:10.795 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:34:10.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:34:10.796 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:34:10.796 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:10.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:10.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:10.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:10.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:34:10.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:34:11.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:34:11.954 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:34:11.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:34:11.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:11.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:11.955 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:11.956 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:12.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:34:12.110 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:34:13.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:34:13.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:34:13.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:34:13.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:13.112 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:13.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:13.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:13.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:34:13.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:34:14.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:34:14.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:34:14.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:34:14.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:14.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:14.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:14.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:14.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:34:14.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:34:15.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:34:15.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:34:15.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:34:15.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:15.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:15.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:15.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:15.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T13:30:37.100168+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:34:15.587 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:34:16.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:34:16.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:34:16.590 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:34:16.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:16.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:16.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:16.591 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:16.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:34:14.589223+0000 '>' 2026-02-17T13:30:37.100168+0000 2026-02-19T13:34:16.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:34:16.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:400: _scrub_abort: perf_counters td/osd-scrub-test 3 2026-02-19T13:34:16.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-19T13:34:16.746 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-19T13:34:16.747 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-19T13:34:16.748 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-19T13:34:16.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:34:16.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-19T13:34:16.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:34:16.823 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:34:16.823 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:34:16.823 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:34:16.823 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:34:16.823 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:34:16.824 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:34:16.824 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:34:16.824 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:34:16.824 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:34:16.824 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:34:16.824 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:34:16.824 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:34:16.824 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:34:16.824 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.824 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.824 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:34:16.824 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:34:16.825 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:34:16.825 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:34:16.825 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:34:16.825 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:34:16.825 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:34:16.825 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:34:16.825 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:34:16.825 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:34:16.825 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:34:16.825 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:34:16.825 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:34:16.825 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.826 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.826 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:34:16.826 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:34:16.826 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:34:16.826 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:34:16.826 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:34:16.826 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:34:16.826 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:34:16.826 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:34:16.826 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:34:16.826 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:34:16.826 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:34:16.826 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:34:16.827 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:34:16.827 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.827 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.827 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:34:16.827 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:34:16.827 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:34:16.827 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:34:16.827 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:34:16.827 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:34:16.827 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:34:16.827 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:34:16.827 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:34:16.828 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:34:16.828 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:34:16.828 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:34:16.828 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:34:16.828 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.828 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.828 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:34:16.828 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:34:16.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:34:16.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-19T13:34:16.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:34:16.898 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:34:16.898 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:34:16.898 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:34:16.898 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:34:16.898 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:34:16.898 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:34:16.898 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:34:16.899 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:34:16.899 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:34:16.899 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:34:16.899 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:34:16.899 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:34:16.899 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:34:16.899 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.899 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.899 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:34:16.899 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:34:16.899 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:34:16.899 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:34:16.900 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:34:16.900 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:34:16.900 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:34:16.900 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:34:16.900 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:34:16.900 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:34:16.900 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:34:16.900 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:34:16.900 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:34:16.900 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.900 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.900 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:34:16.900 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:34:16.901 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:34:16.901 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:34:16.901 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:34:16.901 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:34:16.901 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:34:16.901 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:34:16.901 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:34:16.901 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:34:16.901 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:34:16.901 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:34:16.901 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:34:16.901 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.901 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.902 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:34:16.902 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:34:16.902 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:34:16.902 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:34:16.902 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:34:16.902 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:34:16.902 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:34:16.902 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:34:16.902 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:34:16.902 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 11, 2026-02-19T13:34:16.902 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:34:16.902 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:34:16.902 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:34:16.903 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.903 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.903 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:34:16.903 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:34:16.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:34:16.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-19T13:34:16.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:34:16.968 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:34:16.968 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:34:16.969 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:34:16.969 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:34:16.969 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:34:16.969 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:34:16.969 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:34:16.969 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:34:16.969 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:34:16.969 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:34:16.969 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:34:16.969 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:34:16.969 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:34:16.970 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.970 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.970 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:34:16.970 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:34:16.970 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:34:16.970 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:34:16.970 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:34:16.970 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:34:16.970 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:34:16.970 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:34:16.970 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:34:16.971 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:34:16.971 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:34:16.971 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:34:16.971 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:34:16.971 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.971 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.971 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:34:16.971 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:34:16.971 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:34:16.971 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:34:16.971 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:34:16.971 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:34:16.972 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:34:16.972 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:34:16.972 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:34:16.972 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:34:16.972 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:34:16.972 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:34:16.972 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:34:16.972 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.972 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.972 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:34:16.972 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:34:16.972 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:34:16.973 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:34:16.973 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:34:16.973 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:34:16.973 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:34:16.973 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:34:16.973 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:34:16.973 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:34:16.973 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:34:16.973 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:34:16.973 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:34:16.973 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.973 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:34:16.973 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:34:16.974 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:34:16.974 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Teardown Test TEST_scrub_abort ------------------ 2026-02-19T13:34:16.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_scrub_abort ------------------' 2026-02-19T13:34:16.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-19T13:34:16.975 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:34:16.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:34:16.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:34:16.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:34:16.976 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:34:16.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:34:16.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:34:16.976 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:34:17.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:34:17.079 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:34:17.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:34:17.080 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:34:17.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:34:17.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:34:17.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:34:17.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:34:17.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:34:17.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:34:17.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:34:17.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:34:17.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:34:17.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:34:17.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:34:17.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:34:17.133 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:34:17.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:34:17.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:34:17.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:34:17.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_scrub_abort ------------------' 2026-02-19T13:34:17.134 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Complete Test TEST_scrub_abort ------------------ 2026-02-19T13:34:17.135 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-19T13:34:17.135 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Prepare Test TEST_scrub_permit_time ------------------- 2026-02-19T13:34:17.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_scrub_permit_time -------------------' 2026-02-19T13:34:17.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-19T13:34:17.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-test 2026-02-19T13:34:17.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-test 2026-02-19T13:34:17.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:34:17.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:34:17.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:34:17.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:34:17.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:34:17.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:34:17.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:34:17.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:34:17.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:34:17.138 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:34:17.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:34:17.139 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:34:17.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:34:17.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:34:17.141 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:34:17.142 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:34:17.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:34:17.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:34:17.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:34:17.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:34:17.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:34:17.146 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:34:17.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:34:17.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:34:17.147 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:34:17.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:34:17.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:34:17.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:34:17.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-test 2026-02-19T13:34:17.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:34:17.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:34:17.150 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:34:17.151 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.511299 2026-02-19T13:34:17.152 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:34:17.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:34:17.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:34:17.153 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-19T13:34:17.153 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Run Test TEST_scrub_permit_time ----------------------- 2026-02-19T13:34:17.154 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_scrub_permit_time -----------------------' 2026-02-19T13:34:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_scrub_permit_time td/osd-scrub-test 2026-02-19T13:34:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:414: TEST_scrub_permit_time: local dir=td/osd-scrub-test 2026-02-19T13:34:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:415: TEST_scrub_permit_time: local poolname=test 2026-02-19T13:34:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:416: TEST_scrub_permit_time: local OSDS=3 2026-02-19T13:34:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:417: TEST_scrub_permit_time: local objects=15 2026-02-19T13:34:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:419: TEST_scrub_permit_time: TESTDATA=testdata.511299 2026-02-19T13:34:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:421: TEST_scrub_permit_time: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-19T13:34:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-test 2026-02-19T13:34:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:34:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:34:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:34:17.155 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-test/a 2026-02-19T13:34:17.156 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-19T13:34:17.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:34:17.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:34:17.175 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:34:17.175 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:34:17.176 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:34:17.176 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:34:17.176 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:34:17.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-19T13:34:17.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:34:17.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:34:17.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:34:17.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:34:17.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:34:17.205 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:34:17.206 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:34:17.206 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:34:17.206 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:34:17.206 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:34:17.206 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:34:17.206 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:34:17.206 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-mon.a.asok 2026-02-19T13:34:17.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:34:17.207 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.511299/ceph-mon.a.asok config get fsid 2026-02-19T13:34:17.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:34:17.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:34:17.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:34:17.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:34:17.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:34:17.263 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:34:17.263 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:34:17.263 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:34:17.263 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:34:17.263 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:34:17.264 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:34:17.264 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-mon.a.asok 2026-02-19T13:34:17.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:34:17.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.511299/ceph-mon.a.asok config get mon_host 2026-02-19T13:34:17.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:422: TEST_scrub_permit_time: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-19T13:34:17.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-test 2026-02-19T13:34:17.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:34:17.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:34:17.318 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:34:17.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-test/x 2026-02-19T13:34:17.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:34:17.411 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:34:17.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:34:17.412 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:34:17.412 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:34:17.412 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:34:17.412 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:34:17.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:34:17.413 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:34:17.413 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-19T13:34:17.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:423: TEST_scrub_permit_time: date -d '2 hour ago' +%H 2026-02-19T13:34:17.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:423: TEST_scrub_permit_time: sed 's/^0//' 2026-02-19T13:34:17.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:423: TEST_scrub_permit_time: local scrub_begin_hour=11 2026-02-19T13:34:17.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:424: TEST_scrub_permit_time: date -d '1 hour ago' +%H 2026-02-19T13:34:17.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:424: TEST_scrub_permit_time: sed 's/^0//' 2026-02-19T13:34:17.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:424: TEST_scrub_permit_time: local scrub_end_hour=12 2026-02-19T13:34:17.435 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:425: TEST_scrub_permit_time: expr 3 - 1 2026-02-19T13:34:17.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:425: TEST_scrub_permit_time: seq 0 2 2026-02-19T13:34:17.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:425: TEST_scrub_permit_time: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:34:17.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:427: TEST_scrub_permit_time: run_osd td/osd-scrub-test 0 --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=11 --osd_scrub_end_hour=12 2026-02-19T13:34:17.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:34:17.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:34:17.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:34:17.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:34:17.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-19T13:34:17.439 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:34:17.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:34:17.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:34:17.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:34:17.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-19T13:34:17.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-19T13:34:17.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:34:17.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:34:17.440 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:34:17.440 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:34:17.440 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:34:17.440 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:34:17.440 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:34:17.441 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:34:17.441 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:34:17.441 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:34:17.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:34:17.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:34:17.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:34:17.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:34:17.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:34:17.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:34:17.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:34:17.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:34:17.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:34:17.441 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:34:17.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:34:17.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=11 --osd_scrub_end_hour=12' 2026-02-19T13:34:17.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-19T13:34:17.442 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:34:17.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=32732960-65b8-47e1-8bae-0931fde3a605 2026-02-19T13:34:17.442 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 32732960-65b8-47e1-8bae-0931fde3a605' 2026-02-19T13:34:17.443 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 32732960-65b8-47e1-8bae-0931fde3a605 2026-02-19T13:34:17.443 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:34:17.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBZEZdpB1vzGhAA1Gp8hREVfnWCnpV8OQgmBg== 2026-02-19T13:34:17.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBZEZdpB1vzGhAA1Gp8hREVfnWCnpV8OQgmBg=="}' 2026-02-19T13:34:17.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 32732960-65b8-47e1-8bae-0931fde3a605 -i td/osd-scrub-test/0/new.json 2026-02-19T13:34:17.539 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:34:17.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-19T13:34:17.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=11 --osd_scrub_end_hour=12 --mkfs --key AQBZEZdpB1vzGhAA1Gp8hREVfnWCnpV8OQgmBg== --osd-uuid 32732960-65b8-47e1-8bae-0931fde3a605 2026-02-19T13:34:17.565 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:17.561+0000 7f0477b0da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:17.566 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:17.561+0000 7f0477b0da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:17.567 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:17.565+0000 7f0477b0da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:17.568 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:17.565+0000 7f0477b0da40 -1 bdev(0x56019f91cc00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:34:17.568 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:17.565+0000 7f0477b0da40 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-19T13:34:18.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-19T13:34:18.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:34:18.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:34:18.315 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:34:18.316 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:34:18.518 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:34:18.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:34:18.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=11 --osd_scrub_end_hour=12 2026-02-19T13:34:18.519 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:34:18.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:34:18.520 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:34:18.545 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:18.541+0000 7f3c009a0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:18.547 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:18.545+0000 7f3c009a0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:18.548 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:18.545+0000 7f3c009a0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:18.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:34:18.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:34:18.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:34:18.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:34:18.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:34:18.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:34:18.675 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:34:18.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:34:18.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:34:18.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:34:18.832 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:34:18.916 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:18.913+0000 7f3c009a0a40 -1 Falling back to public interface 2026-02-19T13:34:19.445 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:19.441+0000 7f3c009a0a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:34:19.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:34:19.834 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:34:19.835 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:34:19.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:34:19.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:34:19.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:34:20.001 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:34:21.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:34:21.002 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:34:21.003 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:34:21.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:34:21.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:34:21.004 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:34:21.161 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1028424461,v1:127.0.0.1:6803/1028424461] [v2:127.0.0.1:6804/1028424461,v1:127.0.0.1:6805/1028424461] exists,up 32732960-65b8-47e1-8bae-0931fde3a605 2026-02-19T13:34:21.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:34:21.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:34:21.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:34:21.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:425: TEST_scrub_permit_time: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:34:21.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:427: TEST_scrub_permit_time: run_osd td/osd-scrub-test 1 --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=11 --osd_scrub_end_hour=12 2026-02-19T13:34:21.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:34:21.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:34:21.164 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:34:21.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:34:21.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-19T13:34:21.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:34:21.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:34:21.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:34:21.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:34:21.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-19T13:34:21.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-19T13:34:21.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:34:21.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:34:21.165 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:34:21.165 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:34:21.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:34:21.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:34:21.166 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:34:21.166 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:34:21.166 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:34:21.166 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:34:21.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:34:21.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:34:21.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:34:21.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:34:21.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:34:21.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:34:21.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:34:21.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:34:21.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:34:21.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:34:21.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:34:21.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=11 --osd_scrub_end_hour=12' 2026-02-19T13:34:21.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-19T13:34:21.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:34:21.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=05cd1c4e-4672-4b6c-bada-15b16e3b9944 2026-02-19T13:34:21.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 05cd1c4e-4672-4b6c-bada-15b16e3b9944' 2026-02-19T13:34:21.168 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 05cd1c4e-4672-4b6c-bada-15b16e3b9944 2026-02-19T13:34:21.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:34:21.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBdEZdp9FOxChAA7u8DvU7c6BAx2OnPWB28fQ== 2026-02-19T13:34:21.178 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBdEZdp9FOxChAA7u8DvU7c6BAx2OnPWB28fQ=="}' 2026-02-19T13:34:21.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 05cd1c4e-4672-4b6c-bada-15b16e3b9944 -i td/osd-scrub-test/1/new.json 2026-02-19T13:34:21.330 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:34:21.342 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-19T13:34:21.343 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=11 --osd_scrub_end_hour=12 --mkfs --key AQBdEZdp9FOxChAA7u8DvU7c6BAx2OnPWB28fQ== --osd-uuid 05cd1c4e-4672-4b6c-bada-15b16e3b9944 2026-02-19T13:34:21.366 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:21.361+0000 7f94251eea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:21.368 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:21.365+0000 7f94251eea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:21.369 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:21.365+0000 7f94251eea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:21.370 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:21.365+0000 7f94251eea40 -1 bdev(0x55d2be7d8c00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:34:21.370 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:21.365+0000 7f94251eea40 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-19T13:34:22.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-19T13:34:22.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:34:22.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:34:22.010 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:34:22.010 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:34:22.213 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:34:22.213 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:34:22.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=11 --osd_scrub_end_hour=12 2026-02-19T13:34:22.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:34:22.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:34:22.214 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:34:22.238 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:22.233+0000 7f40b2991a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:22.239 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:22.237+0000 7f40b2991a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:22.240 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:22.237+0000 7f40b2991a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:22.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:34:22.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:34:22.368 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:34:22.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:34:22.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:34:22.369 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:34:22.369 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:34:22.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:34:22.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:34:22.370 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:34:22.527 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:34:22.672 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:22.669+0000 7f40b2991a40 -1 Falling back to public interface 2026-02-19T13:34:23.316 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:23.313+0000 7f40b2991a40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:34:23.528 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:34:23.529 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:34:23.529 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:34:23.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:34:23.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:34:23.530 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:34:23.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:34:24.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:34:24.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:34:24.691 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:34:24.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:34:24.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:34:24.691 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:34:24.849 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/64237410,v1:127.0.0.1:6811/64237410] [v2:127.0.0.1:6812/64237410,v1:127.0.0.1:6813/64237410] exists,up 05cd1c4e-4672-4b6c-bada-15b16e3b9944 2026-02-19T13:34:24.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:34:24.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:34:24.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:34:24.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:425: TEST_scrub_permit_time: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:34:24.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:427: TEST_scrub_permit_time: run_osd td/osd-scrub-test 2 --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=11 --osd_scrub_end_hour=12 2026-02-19T13:34:24.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:34:24.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:34:24.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:34:24.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:34:24.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-19T13:34:24.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:34:24.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:34:24.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:34:24.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:34:24.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-19T13:34:24.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-19T13:34:24.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:34:24.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:34:24.854 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:34:24.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:34:24.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:34:24.854 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:34:24.855 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:34:24.855 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:34:24.855 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:34:24.855 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:34:24.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:34:24.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:34:24.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:34:24.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:34:24.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:34:24.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:34:24.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:34:24.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:34:24.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:34:24.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:34:24.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:34:24.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=11 --osd_scrub_end_hour=12' 2026-02-19T13:34:24.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-19T13:34:24.856 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:34:24.856 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c85b575c-2830-48e3-8659-c9f674f05590 2026-02-19T13:34:24.857 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 c85b575c-2830-48e3-8659-c9f674f05590 2026-02-19T13:34:24.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 c85b575c-2830-48e3-8659-c9f674f05590' 2026-02-19T13:34:24.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:34:24.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBgEZdpcqyeMxAAZEghUHqQxSRgSk6UhhTL+w== 2026-02-19T13:34:24.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBgEZdpcqyeMxAAZEghUHqQxSRgSk6UhhTL+w=="}' 2026-02-19T13:34:24.865 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c85b575c-2830-48e3-8659-c9f674f05590 -i td/osd-scrub-test/2/new.json 2026-02-19T13:34:25.024 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:34:25.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-19T13:34:25.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=11 --osd_scrub_end_hour=12 --mkfs --key AQBgEZdpcqyeMxAAZEghUHqQxSRgSk6UhhTL+w== --osd-uuid c85b575c-2830-48e3-8659-c9f674f05590 2026-02-19T13:34:25.052 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:25.049+0000 7f6a25f8ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:25.054 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:25.049+0000 7f6a25f8ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:25.055 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:25.053+0000 7f6a25f8ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:25.055 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:25.053+0000 7f6a25f8ba40 -1 bdev(0x55f45202ec00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:34:25.055 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:25.053+0000 7f6a25f8ba40 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-19T13:34:25.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-19T13:34:25.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:34:25.673 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T13:34:25.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:34:25.674 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:34:25.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:34:25.878 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T13:34:25.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=11 --osd_scrub_end_hour=12 2026-02-19T13:34:25.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:34:25.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:34:25.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:34:25.903 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:25.901+0000 7f11f4360a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:25.905 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:25.901+0000 7f11f4360a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:25.906 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:25.901+0000 7f11f4360a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:34:26.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:34:26.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:34:26.037 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:34:26.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:34:26.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:34:26.038 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:34:26.038 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:34:26.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:34:26.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:34:26.039 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:34:26.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:34:26.280 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:26.277+0000 7f11f4360a40 -1 Falling back to public interface 2026-02-19T13:34:26.924 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:34:26.921+0000 7f11f4360a40 -1 osd.2 0 log_to_monitors true 2026-02-19T13:34:27.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:34:27.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:34:27.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:34:27.197 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:34:27.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:34:27.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:34:27.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:34:28.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:34:28.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:34:28.358 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:34:28.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:34:28.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:34:28.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:34:28.517 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2506334033,v1:127.0.0.1:6819/2506334033] [v2:127.0.0.1:6820/2506334033,v1:127.0.0.1:6821/2506334033] exists,up c85b575c-2830-48e3-8659-c9f674f05590 2026-02-19T13:34:28.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:34:28.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:34:28.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:34:28.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:435: TEST_scrub_permit_time: create_pool test 1 1 2026-02-19T13:34:28.518 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-19T13:34:28.686 INFO:tasks.workunit.client.0.trial144.stderr:pool 'test' created 2026-02-19T13:34:28.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:34:29.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:436: TEST_scrub_permit_time: wait_for_clean 2026-02-19T13:34:29.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:34:29.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:34:29.698 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:34:29.698 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:34:29.699 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:34:29.699 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:34:29.700 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:34:29.700 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:34:29.700 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:34:29.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:34:29.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:34:29.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:34:29.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:34:29.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:34:29.807 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:34:29.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:34:29.969 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:34:29.969 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:34:29.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:34:29.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:34:29.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:34:30.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-19T13:34:30.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-19T13:34:30.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491' 2026-02-19T13:34:30.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:34:30.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:34:30.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705671 2026-02-19T13:34:30.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705671 2026-02-19T13:34:30.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705671' 2026-02-19T13:34:30.116 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:34:30.116 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:34:30.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542148 2026-02-19T13:34:30.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542148 2026-02-19T13:34:30.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705671 2-60129542148' 2026-02-19T13:34:30.186 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:34:30.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-19T13:34:30.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:34:30.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:34:30.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836491 2026-02-19T13:34:30.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:34:30.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836491 2026-02-19T13:34:30.190 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836491 2026-02-19T13:34:30.190 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-19T13:34:30.190 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:34:30.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T13:34:30.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:34:31.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:34:31.355 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:34:31.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T13:34:31.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:34:32.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-19T13:34:32.514 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:34:32.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836493 -lt 21474836491 2026-02-19T13:34:32.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:34:32.671 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705671 2026-02-19T13:34:32.672 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:34:32.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:34:32.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705671 2026-02-19T13:34:32.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:34:32.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705671 2026-02-19T13:34:32.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-19T13:34:32.676 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705671 2026-02-19T13:34:32.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:34:32.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705673 -lt 38654705671 2026-02-19T13:34:32.833 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:34:32.833 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542148 2026-02-19T13:34:32.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:34:32.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:34:32.835 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542148 2026-02-19T13:34:32.836 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:34:32.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542148 2026-02-19T13:34:32.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-19T13:34:32.838 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542148 2026-02-19T13:34:32.838 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:34:33.003 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542149 -lt 60129542148 2026-02-19T13:34:33.003 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:34:33.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:34:33.004 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:34:33.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:34:33.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:34:33.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:34:33.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:34:33.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:34:33.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:34:33.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:34:33.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:34:33.351 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:34:33.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:34:33.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:34:33.352 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:34:33.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:34:33.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:34:33.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:34:33.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:439: TEST_scrub_permit_time: get_pg test SOMETHING 2026-02-19T13:34:33.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=test 2026-02-19T13:34:33.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=SOMETHING 2026-02-19T13:34:33.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map test SOMETHING 2026-02-19T13:34:33.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-19T13:34:33.701 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:439: TEST_scrub_permit_time: local pgid=1.0 2026-02-19T13:34:33.701 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:440: TEST_scrub_permit_time: get_primary test SOMETHING 2026-02-19T13:34:33.701 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=test 2026-02-19T13:34:33.702 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=SOMETHING 2026-02-19T13:34:33.702 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map test SOMETHING 2026-02-19T13:34:33.702 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:34:33.859 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:440: TEST_scrub_permit_time: local primary=1 2026-02-19T13:34:33.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:441: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:33.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:33.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:33.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:33.860 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:34.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:441: TEST_scrub_permit_time: local last_scrub=2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:34.012 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:446: TEST_scrub_permit_time: ceph tell 1.0 schedule-scrub 86400 2026-02-19T13:34:34.076 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:34:34.076 INFO:tasks.workunit.client.0.trial144.stdout: "deep": false, 2026-02-19T13:34:34.076 INFO:tasks.workunit.client.0.trial144.stdout: "must": false, 2026-02-19T13:34:34.076 INFO:tasks.workunit.client.0.trial144.stdout: "stamp": "2026-02-18T13:34:34.077397+0000" 2026-02-19T13:34:34.076 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:34:34.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i=0 )) 2026-02-19T13:34:34.083 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:34.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:34.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:34.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:34.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:34.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:34.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-19T13:34:28.685199+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:34.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:35.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:35.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:35.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:35.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:35.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:35.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:35.243 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:35.397 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-19T13:34:28.685199+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:35.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:36.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:36.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:36.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:36.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:36.399 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:36.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:36.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:36.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:36.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:37.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:37.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:37.555 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:37.555 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:37.555 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:37.556 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:37.556 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:37.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:37.707 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:38.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:38.708 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:38.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:38.708 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:38.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:38.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:38.709 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:38.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:38.866 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:39.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:39.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:39.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:39.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:39.867 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:39.868 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:39.868 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:40.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:40.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:41.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:41.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:41.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:41.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:41.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:41.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:41.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:41.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:41.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:42.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:42.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:42.182 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:42.182 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:42.182 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:42.183 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:42.183 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:42.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:42.339 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:43.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:43.341 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:43.342 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:43.342 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:43.342 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:43.342 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:43.342 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:43.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:43.497 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:44.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:44.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:44.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:44.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:44.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:44.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:44.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:44.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:44.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:45.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:45.654 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:45.654 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:45.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:45.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:45.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:45.655 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:45.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:45.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:46.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:46.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:46.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:46.812 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:46.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:46.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:46.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:46.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:46.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:47.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:47.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:47.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:47.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:47.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:47.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:47.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:48.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:48.123 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:49.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:49.124 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:49.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:49.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:49.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:49.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:49.125 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:49.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:49.289 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:50.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:50.290 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:50.291 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:50.291 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:50.291 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:50.291 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:50.292 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:50.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:50.454 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:51.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:51.455 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:51.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:51.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:51.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:51.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:51.456 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:51.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:51.614 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:52.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:52.615 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:52.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:52.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:52.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:52.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:52.616 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:52.773 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:52.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:53.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:53.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:53.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:53.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:53.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:53.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:53.776 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:53.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:53.928 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:54.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:54.929 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:54.930 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:54.930 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:54.930 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:54.930 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:54.930 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:55.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:55.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:56.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:56.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:56.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:56.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:56.087 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:56.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:56.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:56.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:56.243 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:57.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:57.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:57.245 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:57.245 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:57.245 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:57.245 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:57.245 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:57.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:57.398 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:58.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:58.399 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:58.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:58.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:58.400 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:58.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:58.401 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:58.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:58.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:34:59.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:34:59.558 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:34:59.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:34:59.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:34:59.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:34:59.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:34:59.559 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:34:59.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:34:59.715 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:35:00.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:35:00.716 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:35:00.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:35:00.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:00.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:35:00.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:00.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:35:00.875 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:35:00.876 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:35:01.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:35:01.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:35:01.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:35:01.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:01.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:35:01.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:01.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:35:02.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:35:02.034 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:35:03.035 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:35:03.036 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:35:03.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:35:03.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:03.036 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:35:03.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:03.037 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:35:03.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:35:03.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:35:04.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:35:04.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:35:04.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:35:04.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:04.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:35:04.195 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:04.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:35:04.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:35:04.354 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:35:05.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:35:05.355 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:35:05.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:35:05.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:05.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:35:05.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:05.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:35:05.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:35:05.513 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:35:06.514 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:35:06.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:35:06.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:35:06.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:06.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:35:06.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:06.516 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:35:06.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:35:06.672 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:35:07.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:35:07.673 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:35:07.674 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-19T13:35:07.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:07.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:35:07.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:07.675 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:35:07.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:450: TEST_scrub_permit_time: test 2026-02-18T13:34:34.077397+0000 '>' 2026-02-19T13:34:28.685199+0000 2026-02-19T13:35:07.830 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:453: TEST_scrub_permit_time: sleep 1 2026-02-19T13:35:08.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i++ )) 2026-02-19T13:35:08.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:449: TEST_scrub_permit_time: (( i < 30 )) 2026-02-19T13:35:08.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: perf_counters td/osd-scrub-test 3 2026-02-19T13:35:08.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-19T13:35:08.831 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-19T13:35:08.832 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-19T13:35:08.834 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-19T13:35:08.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:35:08.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-19T13:35:08.835 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:35:08.908 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:35:08.908 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:35:08.908 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:08.909 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:08.909 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:35:08.909 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:35:08.909 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:08.909 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:08.909 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:08.909 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:08.909 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:08.909 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:08.909 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:08.909 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.910 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.910 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:08.910 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:35:08.910 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:08.910 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:08.910 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:35:08.910 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:35:08.910 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:08.910 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:08.910 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:08.910 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:08.910 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:08.910 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:08.911 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:08.911 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.911 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.911 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:08.911 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:35:08.911 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:08.911 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:08.911 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:35:08.911 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:35:08.911 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:08.911 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:08.911 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:08.912 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:08.912 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:08.912 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:08.912 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:08.912 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.912 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.912 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:08.912 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:35:08.912 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:08.912 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:08.912 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:35:08.912 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:35:08.913 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:08.913 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:08.913 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:08.913 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:08.913 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:08.913 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:08.913 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:08.913 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.913 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.913 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:35:08.913 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:35:08.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:35:08.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-19T13:35:08.914 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:35:08.984 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:35:08.984 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:35:08.984 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:08.985 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:08.985 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:35:08.985 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:35:08.985 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:08.985 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:08.985 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:08.985 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:08.985 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:08.985 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:08.985 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:08.985 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.985 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.986 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:08.986 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:35:08.986 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:08.986 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:08.986 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:35:08.986 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:35:08.986 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:08.986 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:08.986 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:08.986 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:08.986 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:08.986 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:08.987 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:08.987 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.987 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.987 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:08.987 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:35:08.987 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:08.987 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:08.987 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:35:08.987 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:35:08.987 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:08.987 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:08.987 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:08.988 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:08.988 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:08.988 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:08.988 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:08.988 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.988 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.988 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:08.988 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:35:08.988 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:08.988 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:08.988 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:35:08.988 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:35:08.989 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:08.989 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:08.989 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:08.989 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:08.989 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:08.989 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:08.989 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:08.989 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.989 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:08.989 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:35:08.989 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:35:08.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:35:08.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-19T13:35:08.990 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:35:09.057 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:35:09.057 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:35:09.057 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:09.057 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:35:09.058 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_scrub_permit_time ------------------' 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Teardown Test TEST_scrub_permit_time ------------------ 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:35:09.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:35:09.060 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:35:09.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:35:09.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:35:09.060 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:35:09.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:35:09.169 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:35:09.170 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:35:09.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:35:09.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:35:09.171 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:35:09.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:35:09.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:35:09.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:35:09.173 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:35:09.174 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:35:09.175 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:35:09.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:35:09.176 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:35:09.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:35:09.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:35:09.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:35:09.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:35:09.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:35:09.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:35:09.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_scrub_permit_time ------------------' 2026-02-19T13:35:09.189 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Complete Test TEST_scrub_permit_time ------------------ 2026-02-19T13:35:09.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-19T13:35:09.190 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Prepare Test TEST_scrub_test ------------------- 2026-02-19T13:35:09.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_scrub_test -------------------' 2026-02-19T13:35:09.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-19T13:35:09.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-scrub-test 2026-02-19T13:35:09.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-scrub-test 2026-02-19T13:35:09.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:35:09.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:35:09.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:35:09.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:35:09.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:35:09.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:35:09.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:35:09.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:35:09.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:35:09.193 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:35:09.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:35:09.194 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:35:09.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:35:09.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:35:09.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:35:09.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:35:09.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:35:09.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:35:09.198 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:35:09.198 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:35:09.199 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:35:09.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:35:09.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:35:09.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:35:09.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:35:09.201 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:35:09.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:35:09.202 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:35:09.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-scrub-test 2026-02-19T13:35:09.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:35:09.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:35:09.204 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:35:09.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.511299 2026-02-19T13:35:09.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:35:09.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:35:09.206 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:35:09.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-19T13:35:09.207 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Run Test TEST_scrub_test ----------------------- 2026-02-19T13:35:09.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_scrub_test -----------------------' 2026-02-19T13:35:09.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_scrub_test td/osd-scrub-test 2026-02-19T13:35:09.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:52: TEST_scrub_test: local dir=td/osd-scrub-test 2026-02-19T13:35:09.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:53: TEST_scrub_test: local poolname=test 2026-02-19T13:35:09.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:54: TEST_scrub_test: local OSDS=3 2026-02-19T13:35:09.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:55: TEST_scrub_test: local objects=15 2026-02-19T13:35:09.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:57: TEST_scrub_test: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-19T13:35:09.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-scrub-test 2026-02-19T13:35:09.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:35:09.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:35:09.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:35:09.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-scrub-test/a 2026-02-19T13:35:09.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-19T13:35:09.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:35:09.228 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:35:09.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:35:09.229 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:35:09.229 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:35:09.229 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:35:09.229 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:35:09.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-19T13:35:09.256 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:35:09.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:35:09.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:35:09.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:35:09.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:35:09.257 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:35:09.258 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:35:09.258 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:35:09.258 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:35:09.258 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:35:09.258 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:35:09.258 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:35:09.258 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-mon.a.asok 2026-02-19T13:35:09.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:35:09.259 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.511299/ceph-mon.a.asok config get fsid 2026-02-19T13:35:09.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:35:09.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:35:09.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:35:09.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:35:09.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:35:09.316 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:35:09.316 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:35:09.316 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:35:09.316 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:35:09.317 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:35:09.317 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:35:09.317 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.511299/ceph-mon.a.asok 2026-02-19T13:35:09.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:35:09.317 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.511299/ceph-mon.a.asok config get mon_host 2026-02-19T13:35:09.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:58: TEST_scrub_test: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-19T13:35:09.371 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-scrub-test 2026-02-19T13:35:09.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:35:09.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:35:09.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:35:09.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-scrub-test/x 2026-02-19T13:35:09.372 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:35:09.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:35:09.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:35:09.467 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:35:09.468 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:35:09.468 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:35:09.468 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:35:09.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:35:09.468 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:35:09.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-19T13:35:09.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:59: TEST_scrub_test: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 ' 2026-02-19T13:35:09.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:60: TEST_scrub_test: ceph_osd_args+='--osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 ' 2026-02-19T13:35:09.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:61: TEST_scrub_test: ceph_osd_args+=--osd_stats_update_period_scrubbing=2 2026-02-19T13:35:09.487 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:62: TEST_scrub_test: expr 3 - 1 2026-02-19T13:35:09.488 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:62: TEST_scrub_test: seq 0 2 2026-02-19T13:35:09.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:62: TEST_scrub_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:35:09.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:64: TEST_scrub_test: run_osd td/osd-scrub-test 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-19T13:35:09.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:35:09.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:35:09.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:35:09.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:35:09.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-19T13:35:09.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:35:09.492 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:35:09.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:35:09.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:35:09.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-19T13:35:09.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-19T13:35:09.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:35:09.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:35:09.493 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:35:09.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:35:09.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:35:09.493 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:35:09.493 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:35:09.494 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:35:09.494 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:35:09.494 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:35:09.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:35:09.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:35:09.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:35:09.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:35:09.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:35:09.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:35:09.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:35:09.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:35:09.494 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:35:09.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:35:09.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:35:09.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2026-02-19T13:35:09.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-19T13:35:09.495 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:35:09.495 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d591157b-3a4c-40c5-8d6a-1a1eadf2df23 2026-02-19T13:35:09.496 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 d591157b-3a4c-40c5-8d6a-1a1eadf2df23 2026-02-19T13:35:09.496 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d591157b-3a4c-40c5-8d6a-1a1eadf2df23' 2026-02-19T13:35:09.496 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:35:09.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCNEZdp+vIWHhAA4nkGnCY37GHvXhvbG35hRg== 2026-02-19T13:35:09.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCNEZdp+vIWHhAA4nkGnCY37GHvXhvbG35hRg=="}' 2026-02-19T13:35:09.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d591157b-3a4c-40c5-8d6a-1a1eadf2df23 -i td/osd-scrub-test/0/new.json 2026-02-19T13:35:09.594 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:35:09.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-19T13:35:09.603 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQCNEZdp+vIWHhAA4nkGnCY37GHvXhvbG35hRg== --osd-uuid d591157b-3a4c-40c5-8d6a-1a1eadf2df23 2026-02-19T13:35:09.620 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:09.620+0000 7f5e7b33da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:09.622 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:09.620+0000 7f5e7b33da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:09.622 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:09.624+0000 7f5e7b33da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:09.623 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:09.624+0000 7f5e7b33da40 -1 bdev(0x56116194ac00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:35:09.623 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:09.624+0000 7f5e7b33da40 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-19T13:35:10.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-19T13:35:10.373 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:35:10.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:35:10.375 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:35:10.375 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:35:10.579 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:35:10.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:35:10.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-19T13:35:10.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:35:10.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:35:10.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:35:10.605 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:10.604+0000 7fca2dcf4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:10.607 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:10.608+0000 7fca2dcf4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:10.608 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:10.608+0000 7fca2dcf4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:10.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:35:10.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:35:10.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:35:10.741 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:35:10.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:35:10.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:35:10.742 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:35:10.742 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:35:10.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:35:10.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:35:10.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:35:10.996 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:10.996+0000 7fca2dcf4a40 -1 Falling back to public interface 2026-02-19T13:35:11.581 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:11.580+0000 7fca2dcf4a40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:35:11.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:35:11.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:35:11.903 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:35:11.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:35:11.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:35:11.904 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:35:12.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:35:13.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:35:13.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:35:13.071 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:35:13.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:35:13.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:35:13.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:35:13.228 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2927416154,v1:127.0.0.1:6803/2927416154] [v2:127.0.0.1:6804/2927416154,v1:127.0.0.1:6805/2927416154] exists,up d591157b-3a4c-40c5-8d6a-1a1eadf2df23 2026-02-19T13:35:13.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:35:13.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:35:13.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:35:13.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:62: TEST_scrub_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:35:13.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:64: TEST_scrub_test: run_osd td/osd-scrub-test 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-19T13:35:13.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:35:13.231 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:35:13.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:35:13.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:35:13.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-19T13:35:13.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:35:13.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:35:13.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:35:13.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:35:13.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-19T13:35:13.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-19T13:35:13.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:35:13.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:35:13.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:35:13.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2026-02-19T13:35:13.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-19T13:35:13.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:35:13.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d31b9784-1388-4a48-9c23-75e30e0f570d 2026-02-19T13:35:13.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 d31b9784-1388-4a48-9c23-75e30e0f570d' 2026-02-19T13:35:13.234 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 d31b9784-1388-4a48-9c23-75e30e0f570d 2026-02-19T13:35:13.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:35:13.244 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCREZdpeTmiDhAAsn7YgyuUj7+km6LQYXLjdA== 2026-02-19T13:35:13.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCREZdpeTmiDhAAsn7YgyuUj7+km6LQYXLjdA=="}' 2026-02-19T13:35:13.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d31b9784-1388-4a48-9c23-75e30e0f570d -i td/osd-scrub-test/1/new.json 2026-02-19T13:35:13.405 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:35:13.416 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-19T13:35:13.417 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQCREZdpeTmiDhAAsn7YgyuUj7+km6LQYXLjdA== --osd-uuid d31b9784-1388-4a48-9c23-75e30e0f570d 2026-02-19T13:35:13.442 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:13.440+0000 7f000861ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:13.444 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:13.444+0000 7f000861ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:13.445 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:13.444+0000 7f000861ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:13.445 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:13.444+0000 7f000861ca40 -1 bdev(0x5572d9efac00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:35:13.445 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:13.444+0000 7f000861ca40 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-19T13:35:14.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-19T13:35:14.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:35:14.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:35:14.067 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:35:14.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:35:14.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:35:14.269 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:35:14.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-19T13:35:14.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:35:14.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:35:14.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:35:14.294 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:14.292+0000 7f756c111a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:14.295 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:14.296+0000 7f756c111a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:14.296 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:14.296+0000 7f756c111a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:14.428 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:35:14.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:35:14.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:35:14.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:35:14.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:35:14.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:35:14.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:35:14.430 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:35:14.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:35:14.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:35:14.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:35:14.704 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:14.704+0000 7f756c111a40 -1 Falling back to public interface 2026-02-19T13:35:15.254 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:15.252+0000 7f756c111a40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:35:15.592 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:35:15.593 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:35:15.593 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:35:15.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:35:15.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:35:15.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:35:15.753 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:35:16.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:35:16.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:35:16.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:35:16.755 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:35:16.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:35:16.755 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:35:16.911 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/876958373,v1:127.0.0.1:6811/876958373] [v2:127.0.0.1:6812/876958373,v1:127.0.0.1:6813/876958373] exists,up d31b9784-1388-4a48-9c23-75e30e0f570d 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:62: TEST_scrub_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:64: TEST_scrub_test: run_osd td/osd-scrub-test 2 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-scrub-test 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:35:16.912 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:35:16.913 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:35:16.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:35:16.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:35:16.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:35:16.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:35:16.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:35:16.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:35:16.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:35:16.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:35:16.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:35:16.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:35:16.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:35:16.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2026-02-19T13:35:16.913 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-19T13:35:16.915 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:35:16.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3dd7dece-8d64-43ac-a772-175b5319c5d2 2026-02-19T13:35:16.916 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 3dd7dece-8d64-43ac-a772-175b5319c5d2 2026-02-19T13:35:16.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 3dd7dece-8d64-43ac-a772-175b5319c5d2' 2026-02-19T13:35:16.916 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:35:16.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCUEZdpChdYNxAAEY+ZxKuhGZumtGr8y4Q2Pg== 2026-02-19T13:35:16.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCUEZdpChdYNxAAEY+ZxKuhGZumtGr8y4Q2Pg=="}' 2026-02-19T13:35:16.927 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3dd7dece-8d64-43ac-a772-175b5319c5d2 -i td/osd-scrub-test/2/new.json 2026-02-19T13:35:17.089 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:35:17.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-19T13:35:17.102 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQCUEZdpChdYNxAAEY+ZxKuhGZumtGr8y4Q2Pg== --osd-uuid 3dd7dece-8d64-43ac-a772-175b5319c5d2 2026-02-19T13:35:17.126 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:17.124+0000 7f19cd2c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:17.127 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:17.128+0000 7f19cd2c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:17.128 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:17.128+0000 7f19cd2c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:17.129 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:17.128+0000 7f19cd2c7a40 -1 bdev(0x561f6ac88c00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:35:17.129 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:17.128+0000 7f19cd2c7a40 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-19T13:35:17.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-19T13:35:17.766 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:35:17.767 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T13:35:17.767 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:35:17.768 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:35:17.969 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T13:35:17.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:35:17.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-19T13:35:17.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:35:17.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:35:17.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:35:17.995 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:17.996+0000 7f5131518a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:17.997 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:17.996+0000 7f5131518a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:17.999 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:18.000+0000 7f5131518a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:18.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:35:18.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:35:18.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:35:18.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:35:18.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:35:18.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:35:18.126 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:35:18.127 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:35:18.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:35:18.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:35:18.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:35:18.396 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:18.396+0000 7f5131518a40 -1 Falling back to public interface 2026-02-19T13:35:18.936 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:18.936+0000 7f5131518a40 -1 osd.2 0 log_to_monitors true 2026-02-19T13:35:19.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:35:19.286 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:35:19.287 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:35:19.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:35:19.287 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:35:19.288 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:35:19.446 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:35:20.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:35:20.448 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:35:20.448 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:35:20.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:35:20.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:35:20.449 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:35:20.606 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/338299970,v1:127.0.0.1:6819/338299970] [v2:127.0.0.1:6820/338299970,v1:127.0.0.1:6821/338299970] exists,up 3dd7dece-8d64-43ac-a772-175b5319c5d2 2026-02-19T13:35:20.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:35:20.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:35:20.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:35:20.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:68: TEST_scrub_test: create_pool test 1 1 2026-02-19T13:35:20.607 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-19T13:35:20.811 INFO:tasks.workunit.client.0.trial144.stderr:pool 'test' created 2026-02-19T13:35:20.821 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:35:21.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:69: TEST_scrub_test: wait_for_clean 2026-02-19T13:35:21.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:35:21.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:35:21.822 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:35:21.823 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:35:21.824 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:35:21.824 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:35:21.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:35:21.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:35:21.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:35:21.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:35:21.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:35:21.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:35:21.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:35:21.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:35:21.933 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:35:22.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:35:22.092 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:35:22.092 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:35:22.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:35:22.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:22.092 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:35:22.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-19T13:35:22.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-19T13:35:22.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491' 2026-02-19T13:35:22.168 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:22.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:35:22.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705671 2026-02-19T13:35:22.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705671 2026-02-19T13:35:22.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705671' 2026-02-19T13:35:22.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:22.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:35:22.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542148 2026-02-19T13:35:22.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542148 2026-02-19T13:35:22.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705671 2-60129542148' 2026-02-19T13:35:22.311 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:22.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-19T13:35:22.312 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:22.313 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:35:22.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836491 2026-02-19T13:35:22.314 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:22.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836491 2026-02-19T13:35:22.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-19T13:35:22.316 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836491 2026-02-19T13:35:22.316 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:22.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T13:35:22.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:35:23.474 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:35:23.474 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:23.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-19T13:35:23.632 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:35:24.634 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-19T13:35:24.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:24.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836493 -lt 21474836491 2026-02-19T13:35:24.796 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:24.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705671 2026-02-19T13:35:24.797 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:24.798 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:35:24.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705671 2026-02-19T13:35:24.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:24.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705671 2026-02-19T13:35:24.801 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-19T13:35:24.801 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705671 2026-02-19T13:35:24.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:35:24.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705673 -lt 38654705671 2026-02-19T13:35:24.966 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:24.967 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542148 2026-02-19T13:35:24.967 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:24.968 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:35:24.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542148 2026-02-19T13:35:24.969 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:24.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542148 2026-02-19T13:35:24.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-19T13:35:24.971 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542148 2026-02-19T13:35:24.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:35:25.131 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542149 -lt 60129542148 2026-02-19T13:35:25.131 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:35:25.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:35:25.132 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:35:25.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:35:25.324 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:35:25.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:35:25.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:35:25.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:35:25.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:35:25.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:35:25.325 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:35:25.477 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:35:25.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:35:25.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:35:25.478 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:35:25.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:35:25.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:35:25.666 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:35:25.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:70: TEST_scrub_test: ceph osd dump 2026-02-19T13:35:25.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:70: TEST_scrub_test: grep '^pool.*['\'']test['\'']' 2026-02-19T13:35:25.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:70: TEST_scrub_test: awk '{ print $2 }' 2026-02-19T13:35:25.824 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:70: TEST_scrub_test: poolid=1 2026-02-19T13:35:25.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:72: TEST_scrub_test: file_with_random_data 1032 2026-02-19T13:35:25.824 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: local size_bytes=1032 2026-02-19T13:35:25.825 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: mktemp 2026-02-19T13:35:25.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: local file=/tmp/tmp.mc7AFgzj42 2026-02-19T13:35:25.826 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2346: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.mc7AFgzj42 bs=1032 count=1 2026-02-19T13:35:25.827 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:35:25.827 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:35:25.827 INFO:tasks.workunit.client.0.trial144.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 8.3144e-05 s, 12.4 MB/s 2026-02-19T13:35:25.827 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2347: file_with_random_data: printf %s /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:25.827 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:72: TEST_scrub_test: local testdata_file=/tmp/tmp.mc7AFgzj42 2026-02-19T13:35:25.828 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: seq 1 15 2026-02-19T13:35:25.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-19T13:35:25.829 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: rados -p test put obj1 /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:25.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-19T13:35:25.848 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: rados -p test put obj2 /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:25.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-19T13:35:25.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: rados -p test put obj3 /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:25.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-19T13:35:25.899 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: rados -p test put obj4 /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:25.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-19T13:35:25.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: rados -p test put obj5 /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:25.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-19T13:35:25.949 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: rados -p test put obj6 /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:25.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-19T13:35:25.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: rados -p test put obj7 /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:25.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-19T13:35:25.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: rados -p test put obj8 /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:26.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-19T13:35:26.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: rados -p test put obj9 /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:26.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-19T13:35:26.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: rados -p test put obj10 /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:26.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-19T13:35:26.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: rados -p test put obj11 /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:26.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-19T13:35:26.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: rados -p test put obj12 /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:26.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-19T13:35:26.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: rados -p test put obj13 /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:26.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-19T13:35:26.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: rados -p test put obj14 /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:26.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:73: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-19T13:35:26.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: rados -p test put obj15 /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:26.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rm -f /tmp/tmp.mc7AFgzj42 2026-02-19T13:35:26.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:79: TEST_scrub_test: get_primary test obj1 2026-02-19T13:35:26.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=test 2026-02-19T13:35:26.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T13:35:26.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map test obj1 2026-02-19T13:35:26.187 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:35:26.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:79: TEST_scrub_test: local primary=1 2026-02-19T13:35:26.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:80: TEST_scrub_test: get_not_primary test obj1 2026-02-19T13:35:26.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=test 2026-02-19T13:35:26.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=obj1 2026-02-19T13:35:26.346 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary test obj1 2026-02-19T13:35:26.346 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=test 2026-02-19T13:35:26.346 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=obj1 2026-02-19T13:35:26.347 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map test obj1 2026-02-19T13:35:26.347 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:35:26.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-19T13:35:26.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map test obj1 2026-02-19T13:35:26.500 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-19T13:35:26.661 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:80: TEST_scrub_test: local otherosd=0 2026-02-19T13:35:26.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:81: TEST_scrub_test: '[' 0 = 2 ']' 2026-02-19T13:35:26.662 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:85: TEST_scrub_test: local anotherosd=2 2026-02-19T13:35:26.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:88: TEST_scrub_test: file_with_random_data 512 2026-02-19T13:35:26.662 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: local size_bytes=512 2026-02-19T13:35:26.662 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: mktemp 2026-02-19T13:35:26.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: local file=/tmp/tmp.T3g0nI4KxP 2026-02-19T13:35:26.667 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2346: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.T3g0nI4KxP bs=512 count=1 2026-02-19T13:35:26.667 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records in 2026-02-19T13:35:26.668 INFO:tasks.workunit.client.0.trial144.stderr:1+0 records out 2026-02-19T13:35:26.668 INFO:tasks.workunit.client.0.trial144.stderr:512 bytes copied, 8.6791e-05 s, 5.9 MB/s 2026-02-19T13:35:26.668 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2347: file_with_random_data: printf %s /tmp/tmp.T3g0nI4KxP 2026-02-19T13:35:26.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:88: TEST_scrub_test: local corrupt_data_file=/tmp/tmp.T3g0nI4KxP 2026-02-19T13:35:26.668 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:89: TEST_scrub_test: objectstore_tool td/osd-scrub-test 2 obj1 set-bytes /tmp/tmp.T3g0nI4KxP 2026-02-19T13:35:26.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-scrub-test 2026-02-19T13:35:26.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:35:26.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T13:35:26.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:35:26.669 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-test 2 obj1 set-bytes /tmp/tmp.T3g0nI4KxP 2026-02-19T13:35:26.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-scrub-test 2026-02-19T13:35:26.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:35:26.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T13:35:26.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:35:26.670 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-scrub-test TERM osd.2 2026-02-19T13:35:26.670 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:35:26.671 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:35:26.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:35:26.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:35:26.671 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:35:26.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:35:26.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-test 2 obj1 set-bytes /tmp/tmp.T3g0nI4KxP 2026-02-19T13:35:26.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-scrub-test 2026-02-19T13:35:26.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:35:26.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T13:35:26.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:35:26.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-scrub-test/2 2026-02-19T13:35:26.974 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-test/2 obj1 set-bytes /tmp/tmp.T3g0nI4KxP 2026-02-19T13:35:27.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-scrub-test 2 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-19T13:35:27.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-scrub-test 2026-02-19T13:35:27.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:35:27.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:35:27.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:35:27.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-scrub-test/2 2026-02-19T13:35:27.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-19T13:35:27.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:35:27.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:35:27.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:35:27.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-19T13:35:27.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-19T13:35:27.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:35:27.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:35:27.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-19T13:35:27.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:35:27.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:35:27.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:35:27.260 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:35:27.260 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:35:27.260 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:35:27.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:35:27.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' 2026-02-19T13:35:27.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:35:27.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-19T13:35:27.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-19T13:35:27.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:35:27.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:35:27.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:35:27.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:35:27.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:35:27.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2026-02-19T13:35:27.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-scrub-test/2 2026-02-19T13:35:27.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:35:27.262 INFO:tasks.workunit.client.0.trial144.stderr:start osd.2 2026-02-19T13:35:27.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=3d295459-d27a-4514-9ecb-e339be34cc75 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.511299/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-19T13:35:27.262 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-scrub-test/2/whoami 2026-02-19T13:35:27.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:35:27.263 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:35:27.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:35:27.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:35:27.282 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:27.284+0000 7f6eca44ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:27.284 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:27.284+0000 7f6eca44ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:27.285 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:27.284+0000 7f6eca44ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:35:27.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:35:27.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:35:27.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:35:27.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:35:27.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:35:27.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:35:27.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:35:27.423 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:35:27.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:35:27.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:35:27.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:35:27.708 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:27.708+0000 7f6eca44ca40 -1 Falling back to public interface 2026-02-19T13:35:28.300 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:35:28.300+0000 7f6eca44ca40 -1 osd.2 17 log_to_monitors true 2026-02-19T13:35:28.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:35:28.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 500 )) 2026-02-19T13:35:28.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:35:28.583 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:35:28.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:35:28.583 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:35:28.739 INFO:tasks.workunit.client.0.trial144.stderr:osd.2 up in weight 1 up_from 21 up_thru 0 down_at 18 last_clean_interval [14,17) [v2:127.0.0.1:6818/3698608730,v1:127.0.0.1:6819/3698608730] [v2:127.0.0.1:6820/3698608730,v1:127.0.0.1:6821/3698608730] exists,up 3dd7dece-8d64-43ac-a772-175b5319c5d2 2026-02-19T13:35:28.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:35:28.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:35:28.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:35:28.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:35:28.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:35:28.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:35:28.739 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:35:28.740 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:35:28.740 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:35:28.741 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:35:28.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:35:28.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:35:28.741 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:35:28.842 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:35:28.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:35:28.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:35:28.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:35:28.843 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:35:28.843 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:35:29.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:35:29.007 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:35:29.007 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:35:29.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:35:29.007 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:29.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:35:29.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836499 2026-02-19T13:35:29.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836499 2026-02-19T13:35:29.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499' 2026-02-19T13:35:29.078 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:29.078 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:35:29.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705679 2026-02-19T13:35:29.148 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705679 2026-02-19T13:35:29.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705679' 2026-02-19T13:35:29.149 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:29.149 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:35:29.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313218 2026-02-19T13:35:29.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313218 2026-02-19T13:35:29.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705679 2-90194313218' 2026-02-19T13:35:29.222 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:29.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836499 2026-02-19T13:35:29.223 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:29.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:35:29.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836499 2026-02-19T13:35:29.224 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:29.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836499 2026-02-19T13:35:29.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2026-02-19T13:35:29.226 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836499 2026-02-19T13:35:29.226 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:29.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836497 -lt 21474836499 2026-02-19T13:35:29.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:35:30.392 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:35:30.392 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:30.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836500 -lt 21474836499 2026-02-19T13:35:30.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:30.553 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705679 2026-02-19T13:35:30.553 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:30.554 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:35:30.555 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705679 2026-02-19T13:35:30.555 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:30.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705679 2026-02-19T13:35:30.557 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705679' 2026-02-19T13:35:30.557 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 38654705679 2026-02-19T13:35:30.557 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:35:30.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705680 -lt 38654705679 2026-02-19T13:35:30.723 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:30.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-90194313218 2026-02-19T13:35:30.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:30.725 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:35:30.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-90194313218 2026-02-19T13:35:30.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:30.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313218 2026-02-19T13:35:30.727 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 90194313218' 2026-02-19T13:35:30.727 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 90194313218 2026-02-19T13:35:30.728 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:35:30.885 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313219 -lt 90194313218 2026-02-19T13:35:30.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:35:30.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:35:30.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:35:31.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:35:31.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:35:31.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:35:31.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:35:31.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:35:31.082 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:35:31.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:35:31.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:35:31.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:35:31.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:35:31.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:35:31.238 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:35:31.425 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:35:31.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:35:31.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:35:31.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:90: TEST_scrub_test: rm -f /tmp/tmp.T3g0nI4KxP 2026-02-19T13:35:31.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:92: TEST_scrub_test: local pgid=1.0 2026-02-19T13:35:31.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:93: TEST_scrub_test: pg_deep_scrub 1.0 2026-02-19T13:35:31.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_deep_scrub: local pgid=1.0 2026-02-19T13:35:31.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_deep_scrub: wait_for_pg_clean 1.0 2026-02-19T13:35:31.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-19T13:35:31.427 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:35:31.428 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:35:31.428 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:35:31.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:35:31.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:35:31.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:35:31.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:35:31.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:35:31.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:35:31.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:35:31.720 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:35:31.720 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:35:31.878 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:35:31.878 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:35:31.879 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:35:31.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:35:31.879 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:31.879 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:35:31.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836503 2026-02-19T13:35:31.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836503 2026-02-19T13:35:31.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503' 2026-02-19T13:35:31.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:31.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:35:32.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705683 2026-02-19T13:35:32.020 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705683 2026-02-19T13:35:32.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-38654705683' 2026-02-19T13:35:32.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:32.021 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:35:32.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313222 2026-02-19T13:35:32.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313222 2026-02-19T13:35:32.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-38654705683 2-90194313222' 2026-02-19T13:35:32.093 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:32.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836503 2026-02-19T13:35:32.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:32.094 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:35:32.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836503 2026-02-19T13:35:32.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:32.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836503 2026-02-19T13:35:32.097 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836503' 2026-02-19T13:35:32.097 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836503 2026-02-19T13:35:32.098 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:32.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836503 -lt 21474836503 2026-02-19T13:35:32.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:32.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705683 2026-02-19T13:35:32.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:32.266 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:35:32.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705683 2026-02-19T13:35:32.267 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:32.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705683 2026-02-19T13:35:32.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705683' 2026-02-19T13:35:32.269 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705683 2026-02-19T13:35:32.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:35:32.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705682 -lt 38654705683 2026-02-19T13:35:32.422 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:35:33.423 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:35:33.424 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:35:33.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705682 -lt 38654705683 2026-02-19T13:35:33.584 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:35:34.585 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-19T13:35:34.586 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:35:34.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705685 -lt 38654705683 2026-02-19T13:35:34.743 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:34.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-90194313222 2026-02-19T13:35:34.743 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:34.745 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:35:34.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-90194313222 2026-02-19T13:35:34.746 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:34.747 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313222 2026-02-19T13:35:34.747 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 90194313222 2026-02-19T13:35:34.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 90194313222' 2026-02-19T13:35:34.748 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:35:34.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313224 -lt 90194313222 2026-02-19T13:35:34.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:35:34.907 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 1.0 loop 0 2026-02-19T13:35:34.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-19T13:35:34.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-19T13:35:34.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-19T13:35:34.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:35:34.908 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-19T13:35:34.908 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:35:34.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-19T13:35:34.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:35:34.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:35:34.982 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:35:34.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-19T13:35:34.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:34.982 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:35:34.983 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:34.983 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-19T13:35:35.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: local last_scrub=2026-02-19T13:35:20.812357+0000 2026-02-19T13:35:35.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_deep_scrub: ceph pg deep-scrub 1.0 2026-02-19T13:35:35.286 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.0 on osd.1 to deep-scrub 2026-02-19T13:35:35.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_deep_scrub: wait_for_scrub 1.0 2026-02-19T13:35:20.812357+0000 last_deep_scrub_stamp 2026-02-19T13:35:35.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T13:35:35.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:35:20.812357+0000 2026-02-19T13:35:35.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-19T13:35:35.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:35:35.296 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:35:35.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-19T13:35:35.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:35.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:35:35.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:35.297 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-19T13:35:35.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:35:20.812357+0000 '>' 2026-02-19T13:35:20.812357+0000 2026-02-19T13:35:35.457 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:35:36.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:35:36.458 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:35:36.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-19T13:35:36.459 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:36.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:35:36.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-19T13:35:36.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:36.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:35:20.812357+0000 '>' 2026-02-19T13:35:20.812357+0000 2026-02-19T13:35:36.619 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:35:37.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:35:37.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:35:37.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-19T13:35:37.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:37.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:35:37.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:37.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-19T13:35:37.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:35:20.812357+0000 '>' 2026-02-19T13:35:20.812357+0000 2026-02-19T13:35:37.775 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:35:38.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:35:38.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:35:38.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-19T13:35:38.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:38.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:35:38.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:38.778 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-19T13:35:38.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:35:35.441291+0000 '>' 2026-02-19T13:35:20.812357+0000 2026-02-19T13:35:38.932 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:35:38.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:95: TEST_scrub_test: ceph pg dump pgs 2026-02-19T13:35:38.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:95: TEST_scrub_test: grep '^1.0' 2026-02-19T13:35:38.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:95: TEST_scrub_test: grep -q -- +inconsistent 2026-02-19T13:35:39.081 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:35:39.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:96: TEST_scrub_test: ceph pg 1.0 query 2026-02-19T13:35:39.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:96: TEST_scrub_test: jq .info.stats.stat_sum.num_scrub_errors 2026-02-19T13:35:39.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:96: TEST_scrub_test: test 2 = 2 2026-02-19T13:35:39.169 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:98: TEST_scrub_test: ceph osd out 1 2026-02-19T13:35:39.374 INFO:tasks.workunit.client.0.trial144.stderr:marked out osd.1. 2026-02-19T13:35:39.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:99: TEST_scrub_test: wait_for_clean 2026-02-19T13:35:39.389 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:35:39.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:35:39.390 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:35:39.390 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:35:39.391 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:35:39.391 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:35:39.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:35:39.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:35:39.391 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:35:39.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:35:39.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:35:39.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:35:39.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:35:39.498 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:35:39.499 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:35:39.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:35:39.657 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:35:39.657 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:35:39.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:35:39.657 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:39.657 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:35:39.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836511 2026-02-19T13:35:39.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836511 2026-02-19T13:35:39.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836511' 2026-02-19T13:35:39.729 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:39.729 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:35:39.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705692 2026-02-19T13:35:39.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705692 2026-02-19T13:35:39.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836511 1-38654705692' 2026-02-19T13:35:39.800 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:39.801 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:35:39.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313231 2026-02-19T13:35:39.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313231 2026-02-19T13:35:39.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836511 1-38654705692 2-90194313231' 2026-02-19T13:35:39.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:39.873 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836511 2026-02-19T13:35:39.873 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:39.874 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:35:39.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836511 2026-02-19T13:35:39.875 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:39.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836511 2026-02-19T13:35:39.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836511' 2026-02-19T13:35:39.877 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836511 2026-02-19T13:35:39.878 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:40.041 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836512 -lt 21474836511 2026-02-19T13:35:40.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:40.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705692 2026-02-19T13:35:40.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:40.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:35:40.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705692 2026-02-19T13:35:40.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:40.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705692 2026-02-19T13:35:40.046 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705692 2026-02-19T13:35:40.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705692' 2026-02-19T13:35:40.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:35:40.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705692 -lt 38654705692 2026-02-19T13:35:40.205 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:40.205 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-90194313231 2026-02-19T13:35:40.206 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:40.207 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:35:40.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-90194313231 2026-02-19T13:35:40.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:40.209 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313231 2026-02-19T13:35:40.209 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 90194313231 2026-02-19T13:35:40.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 90194313231' 2026-02-19T13:35:40.210 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:35:40.378 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313231 -lt 90194313231 2026-02-19T13:35:40.379 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:35:40.379 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:35:40.379 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:35:40.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:35:40.574 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:35:40.574 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:35:40.574 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:35:40.574 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:35:40.574 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:35:40.575 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:35:40.575 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:35:40.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:35:40.734 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:35:40.734 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:35:40.734 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:35:40.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:35:40.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:35:40.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:35:40.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:101: TEST_scrub_test: pg_deep_scrub 1.0 2026-02-19T13:35:40.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_deep_scrub: local pgid=1.0 2026-02-19T13:35:40.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_deep_scrub: wait_for_pg_clean 1.0 2026-02-19T13:35:40.925 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-19T13:35:40.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-19T13:35:40.926 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:35:40.926 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:35:40.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:35:40.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:35:40.927 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:35:41.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-19T13:35:41.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-19T13:35:41.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-19T13:35:41.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-19T13:35:41.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:35:41.203 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:35:41.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:35:41.361 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:35:41.361 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:35:41.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:35:41.361 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:41.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:35:41.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836514 2026-02-19T13:35:41.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836514 2026-02-19T13:35:41.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514' 2026-02-19T13:35:41.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:41.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:35:41.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705695 2026-02-19T13:35:41.504 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705695 2026-02-19T13:35:41.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-38654705695' 2026-02-19T13:35:41.505 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:41.505 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:35:41.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313234 2026-02-19T13:35:41.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313234 2026-02-19T13:35:41.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-38654705695 2-90194313234' 2026-02-19T13:35:41.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:41.578 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836514 2026-02-19T13:35:41.579 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:41.580 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:35:41.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836514 2026-02-19T13:35:41.580 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:41.581 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836514 2026-02-19T13:35:41.582 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836514 2026-02-19T13:35:41.582 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836514' 2026-02-19T13:35:41.582 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:41.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836512 -lt 21474836514 2026-02-19T13:35:41.748 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:35:42.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:35:42.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:42.907 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836515 -lt 21474836514 2026-02-19T13:35:42.908 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:42.908 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705695 2026-02-19T13:35:42.908 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:42.909 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:35:42.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705695 2026-02-19T13:35:42.910 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:42.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705695 2026-02-19T13:35:42.912 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705695' 2026-02-19T13:35:42.912 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705695 2026-02-19T13:35:42.912 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:35:43.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705695 -lt 38654705695 2026-02-19T13:35:43.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:43.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-90194313234 2026-02-19T13:35:43.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:43.082 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:35:43.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-90194313234 2026-02-19T13:35:43.083 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:43.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313234 2026-02-19T13:35:43.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 90194313234' 2026-02-19T13:35:43.085 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 90194313234 2026-02-19T13:35:43.085 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:35:43.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313234 -lt 90194313234 2026-02-19T13:35:43.240 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-19T13:35:43.240 INFO:tasks.workunit.client.0.trial144.stdout:#---------- 1.0 loop 0 2026-02-19T13:35:43.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-19T13:35:43.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-19T13:35:43.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-19T13:35:43.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-19T13:35:43.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-19T13:35:43.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-19T13:35:43.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean+remapped 2026-02-19T13:35:43.314 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean+remapped == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-19T13:35:43.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-19T13:35:43.315 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-19T13:35:43.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-19T13:35:43.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:43.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:35:43.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-19T13:35:43.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:43.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_deep_scrub: local last_scrub=2026-02-19T13:35:35.441291+0000 2026-02-19T13:35:43.467 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_deep_scrub: ceph pg deep-scrub 1.0 2026-02-19T13:35:43.610 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.0 on osd.0 to deep-scrub 2026-02-19T13:35:43.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_deep_scrub: wait_for_scrub 1.0 2026-02-19T13:35:35.441291+0000 last_deep_scrub_stamp 2026-02-19T13:35:43.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T13:35:43.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:35:35.441291+0000 2026-02-19T13:35:43.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-19T13:35:43.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:35:43.621 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:35:43.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-19T13:35:43.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:43.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:35:43.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:43.622 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-19T13:35:43.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:35:35.441291+0000 '>' 2026-02-19T13:35:35.441291+0000 2026-02-19T13:35:43.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:35:44.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:35:44.778 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:35:44.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-19T13:35:44.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:44.779 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:35:44.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:44.780 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-19T13:35:44.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:35:35.441291+0000 '>' 2026-02-19T13:35:35.441291+0000 2026-02-19T13:35:44.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:35:45.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:35:45.937 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:35:45.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-19T13:35:45.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:45.938 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-19T13:35:45.939 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:45.939 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-19T13:35:46.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:35:44.485704+0000 '>' 2026-02-19T13:35:35.441291+0000 2026-02-19T13:35:46.095 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:35:46.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:103: TEST_scrub_test: ceph pg 1.0 query 2026-02-19T13:35:46.095 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:103: TEST_scrub_test: jq .info.stats.stat_sum.num_scrub_errors 2026-02-19T13:35:46.166 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:103: TEST_scrub_test: test 2 = 2 2026-02-19T13:35:46.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:104: TEST_scrub_test: ceph pg 1.0 query 2026-02-19T13:35:46.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:104: TEST_scrub_test: jq '.peer_info[0].stats.stat_sum.num_scrub_errors' 2026-02-19T13:35:46.241 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:104: TEST_scrub_test: test 2 = 2 2026-02-19T13:35:46.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:105: TEST_scrub_test: ceph pg dump pgs 2026-02-19T13:35:46.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:105: TEST_scrub_test: grep '^1.0' 2026-02-19T13:35:46.242 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:105: TEST_scrub_test: grep -q -- +inconsistent 2026-02-19T13:35:46.386 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:35:46.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:107: TEST_scrub_test: ceph osd in 1 2026-02-19T13:35:46.644 INFO:tasks.workunit.client.0.trial144.stderr:marked in osd.1. 2026-02-19T13:35:46.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:108: TEST_scrub_test: wait_for_clean 2026-02-19T13:35:46.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:35:46.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:35:46.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:35:46.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:35:46.657 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:35:46.657 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:35:46.657 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:35:46.658 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:35:46.658 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:35:46.776 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:35:46.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:35:46.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:35:46.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:35:46.777 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:35:46.777 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:35:46.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:35:46.940 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:35:46.940 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:35:46.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:35:46.941 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:46.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:35:47.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836521 2026-02-19T13:35:47.013 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836521 2026-02-19T13:35:47.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521' 2026-02-19T13:35:47.014 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:47.014 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:35:47.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705701 2026-02-19T13:35:47.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705701 2026-02-19T13:35:47.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-38654705701' 2026-02-19T13:35:47.085 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:47.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:35:47.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313240 2026-02-19T13:35:47.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313240 2026-02-19T13:35:47.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-38654705701 2-90194313240' 2026-02-19T13:35:47.159 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:47.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836521 2026-02-19T13:35:47.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:47.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:35:47.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836521 2026-02-19T13:35:47.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:47.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836521 2026-02-19T13:35:47.163 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836521' 2026-02-19T13:35:47.163 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836521 2026-02-19T13:35:47.163 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:47.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836519 -lt 21474836521 2026-02-19T13:35:47.319 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:35:48.320 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:35:48.320 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:48.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836522 -lt 21474836521 2026-02-19T13:35:48.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:48.481 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705701 2026-02-19T13:35:48.482 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:48.483 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:35:48.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705701 2026-02-19T13:35:48.484 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:48.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705701 2026-02-19T13:35:48.486 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705701' 2026-02-19T13:35:48.486 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705701 2026-02-19T13:35:48.486 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:35:48.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705702 -lt 38654705701 2026-02-19T13:35:48.651 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:48.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-90194313240 2026-02-19T13:35:48.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:48.653 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:35:48.654 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-90194313240 2026-02-19T13:35:48.654 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:48.655 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313240 2026-02-19T13:35:48.655 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 90194313240 2026-02-19T13:35:48.656 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 90194313240' 2026-02-19T13:35:48.656 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:35:48.815 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313241 -lt 90194313240 2026-02-19T13:35:48.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:35:48.815 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:35:48.816 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:35:49.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:35:49.009 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:35:49.009 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:35:49.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:35:49.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:35:49.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:35:49.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:35:49.010 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:35:49.167 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:35:49.167 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:35:49.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:35:49.168 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:35:49.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:35:49.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:35:49.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:35:49.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:110: TEST_scrub_test: repair 1.0 2026-02-19T13:35:49.360 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1887: repair: local pgid=1.0 2026-02-19T13:35:49.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: get_last_scrub_stamp 1.0 2026-02-19T13:35:49.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:49.360 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:35:49.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:49.361 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:35:49.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1888: repair: local last_scrub=2026-02-19T13:35:44.485704+0000 2026-02-19T13:35:49.521 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1889: repair: ceph pg repair 1.0 2026-02-19T13:35:49.665 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.0 on osd.1 to repair 2026-02-19T13:35:49.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1890: repair: wait_for_scrub 1.0 2026-02-19T13:35:44.485704+0000 2026-02-19T13:35:49.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-19T13:35:49.675 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-19T13:35:44.485704+0000 2026-02-19T13:35:49.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-19T13:35:49.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-19T13:35:49.676 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:35:49.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:35:49.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:49.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:35:49.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:49.676 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:35:49.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:35:44.485704+0000 '>' 2026-02-19T13:35:44.485704+0000 2026-02-19T13:35:49.837 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:35:50.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:35:50.838 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:35:50.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:35:50.839 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:50.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:35:50.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:50.840 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:35:50.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:35:44.485704+0000 '>' 2026-02-19T13:35:44.485704+0000 2026-02-19T13:35:50.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-19T13:35:52.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-19T13:35:52.000 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 500 )) 2026-02-19T13:35:52.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-19T13:35:52.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-19T13:35:52.000 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-19T13:35:52.001 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-19T13:35:52.001 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-19T13:35:52.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-19T13:35:50.482384+0000 '>' 2026-02-19T13:35:44.485704+0000 2026-02-19T13:35:52.157 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-19T13:35:52.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:111: TEST_scrub_test: wait_for_clean 2026-02-19T13:35:52.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:35:52.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:35:52.158 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:35:52.158 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:35:52.159 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:35:52.159 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:35:52.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:35:52.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:35:52.159 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:35:52.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:35:52.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:35:52.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:35:52.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:35:52.259 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:35:52.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:35:52.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:35:52.415 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:35:52.415 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:35:52.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:35:52.415 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:52.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:35:52.488 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836527 2026-02-19T13:35:52.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836527 2026-02-19T13:35:52.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836527' 2026-02-19T13:35:52.489 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:52.489 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:35:52.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705708 2026-02-19T13:35:52.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705708 2026-02-19T13:35:52.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836527 1-38654705708' 2026-02-19T13:35:52.563 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:52.563 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:35:52.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313247 2026-02-19T13:35:52.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313247 2026-02-19T13:35:52.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836527 1-38654705708 2-90194313247' 2026-02-19T13:35:52.633 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:52.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836527 2026-02-19T13:35:52.634 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:52.635 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:35:52.636 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836527 2026-02-19T13:35:52.636 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:52.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836527 2026-02-19T13:35:52.638 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836527' 2026-02-19T13:35:52.638 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836527 2026-02-19T13:35:52.638 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:52.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836526 -lt 21474836527 2026-02-19T13:35:52.806 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:35:53.807 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:35:53.808 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:53.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836526 -lt 21474836527 2026-02-19T13:35:53.969 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:35:54.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-19T13:35:54.971 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:55.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836529 -lt 21474836527 2026-02-19T13:35:55.128 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:55.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705708 2026-02-19T13:35:55.128 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:55.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:35:55.131 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705708 2026-02-19T13:35:55.131 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:55.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705708 2026-02-19T13:35:55.132 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705708 2026-02-19T13:35:55.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705708' 2026-02-19T13:35:55.133 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:35:55.299 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705709 -lt 38654705708 2026-02-19T13:35:55.300 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:55.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-90194313247 2026-02-19T13:35:55.300 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:55.301 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:35:55.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-90194313247 2026-02-19T13:35:55.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:55.303 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313247 2026-02-19T13:35:55.304 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 90194313247' 2026-02-19T13:35:55.304 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 90194313247 2026-02-19T13:35:55.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:35:55.460 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313248 -lt 90194313247 2026-02-19T13:35:55.460 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:35:55.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:35:55.461 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:35:55.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:35:55.652 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:35:55.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:35:55.652 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:35:55.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:35:55.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:35:55.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:35:55.653 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:35:55.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:35:55.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:35:55.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:35:55.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:35:55.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:35:55.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:35:55.997 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:35:55.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:114: TEST_scrub_test: ceph pg 1.0 query 2026-02-19T13:35:55.997 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:114: TEST_scrub_test: jq '.peer_info[0].stats.stat_sum.num_scrub_errors' 2026-02-19T13:35:56.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:114: TEST_scrub_test: test 2 = 2 2026-02-19T13:35:56.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:115: TEST_scrub_test: ceph pg dump pgs 2026-02-19T13:35:56.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:115: TEST_scrub_test: grep '^1.0' 2026-02-19T13:35:56.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:115: TEST_scrub_test: grep -vq -- +inconsistent 2026-02-19T13:35:56.215 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:35:56.224 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:117: TEST_scrub_test: ceph osd out 1 2026-02-19T13:35:56.424 INFO:tasks.workunit.client.0.trial144.stderr:marked out osd.1. 2026-02-19T13:35:56.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:118: TEST_scrub_test: wait_for_clean 2026-02-19T13:35:56.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:35:56.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:35:56.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:35:56.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:35:56.435 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:35:56.435 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:35:56.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:35:56.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:35:56.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:35:56.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:35:56.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:35:56.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:35:56.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:35:56.555 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=500 2026-02-19T13:35:56.555 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:35:56.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:35:56.711 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:35:56.711 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:35:56.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:35:56.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:56.711 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:35:56.784 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836533 2026-02-19T13:35:56.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836533 2026-02-19T13:35:56.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836533' 2026-02-19T13:35:56.785 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:56.785 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:35:56.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705713 2026-02-19T13:35:56.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705713 2026-02-19T13:35:56.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836533 1-38654705713' 2026-02-19T13:35:56.857 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:35:56.857 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:35:56.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313252 2026-02-19T13:35:56.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313252 2026-02-19T13:35:56.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836533 1-38654705713 2-90194313252' 2026-02-19T13:35:56.931 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:56.931 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836533 2026-02-19T13:35:56.932 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:56.933 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:35:56.934 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836533 2026-02-19T13:35:56.934 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:56.935 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836533 2026-02-19T13:35:56.935 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836533 2026-02-19T13:35:56.936 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836533' 2026-02-19T13:35:56.936 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:57.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836531 -lt 21474836533 2026-02-19T13:35:57.100 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:35:58.101 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-19T13:35:58.102 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:35:58.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836534 -lt 21474836533 2026-02-19T13:35:58.265 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:58.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705713 2026-02-19T13:35:58.266 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:58.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:35:58.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705713 2026-02-19T13:35:58.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:58.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705713 2026-02-19T13:35:58.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705713' 2026-02-19T13:35:58.269 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 38654705713 2026-02-19T13:35:58.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:35:58.433 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705714 -lt 38654705713 2026-02-19T13:35:58.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:35:58.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-90194313252 2026-02-19T13:35:58.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:35:58.436 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:35:58.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-90194313252 2026-02-19T13:35:58.436 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:35:58.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313252 2026-02-19T13:35:58.438 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 90194313252' 2026-02-19T13:35:58.439 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 90194313252 2026-02-19T13:35:58.439 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:35:58.594 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313253 -lt 90194313252 2026-02-19T13:35:58.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:35:58.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:35:58.595 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:35:58.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:35:58.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:35:58.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:35:58.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:35:58.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:35:58.783 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:35:58.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:35:58.784 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:35:58.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:35:58.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:35:58.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:35:58.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:35:59.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:35:59.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:35:59.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:35:59.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:120: TEST_scrub_test: ceph pg 1.0 query 2026-02-19T13:35:59.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:120: TEST_scrub_test: jq .info.stats.stat_sum.num_scrub_errors 2026-02-19T13:35:59.212 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:120: TEST_scrub_test: test 0 = 0 2026-02-19T13:35:59.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:121: TEST_scrub_test: ceph pg 1.0 query 2026-02-19T13:35:59.213 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:121: TEST_scrub_test: jq '.peer_info[0].stats.stat_sum.num_scrub_errors' 2026-02-19T13:35:59.285 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:121: TEST_scrub_test: test 0 = 0 2026-02-19T13:35:59.286 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:122: TEST_scrub_test: ceph pg 1.0 query 2026-02-19T13:35:59.287 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:122: TEST_scrub_test: jq '.peer_info[1].stats.stat_sum.num_scrub_errors' 2026-02-19T13:35:59.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:122: TEST_scrub_test: test 0 = 0 2026-02-19T13:35:59.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:123: TEST_scrub_test: ceph pg dump pgs 2026-02-19T13:35:59.358 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:123: TEST_scrub_test: grep '^1.0' 2026-02-19T13:35:59.359 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:123: TEST_scrub_test: grep -vq -- +inconsistent 2026-02-19T13:35:59.503 INFO:tasks.workunit.client.0.trial144.stderr:dumped pgs 2026-02-19T13:35:59.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:124: TEST_scrub_test: perf_counters td/osd-scrub-test 3 2026-02-19T13:35:59.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-19T13:35:59.512 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-19T13:35:59.513 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-19T13:35:59.514 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-19T13:35:59.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:35:59.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-19T13:35:59.515 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:35:59.589 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:35:59.590 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:35:59.590 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:59.590 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:59.590 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:35:59.590 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:35:59.591 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:59.591 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:59.591 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:59.591 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:59.591 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:59.592 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:59.592 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:59.592 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.592 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.592 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:59.593 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:35:59.593 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:59.593 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:59.593 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:35:59.593 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:35:59.593 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:59.594 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:59.594 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:59.594 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 2, 2026-02-19T13:35:59.594 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:59.594 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:59.594 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:59.595 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.595 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.595 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:59.595 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:35:59.595 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:59.596 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:59.596 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:35:59.596 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:35:59.596 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:59.596 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:59.596 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:59.597 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:59.597 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:59.597 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:59.597 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:59.597 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.597 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.598 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:59.598 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:35:59.598 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:59.598 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:59.598 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:35:59.599 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:35:59.599 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:59.599 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:59.599 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:59.599 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:59.599 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:59.600 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:59.600 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:59.600 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.600 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.600 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:35:59.600 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:35:59.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:35:59.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-19T13:35:59.602 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:35:59.660 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 6, 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:35:59.661 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:35:59.662 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:35:59.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-19T13:35:59.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-19T13:35:59.663 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-19T13:35:59.731 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:35:59.731 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_ec": [ 2026-02-19T13:35:59.731 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:59.731 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:59.731 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:35:59.731 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:35:59.731 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:59.731 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:59.731 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_dp_repl": [ 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: "level": "deep", 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:59.732 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_ec": [ 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "ec" 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: ], 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: "osd_scrub_sh_repl": [ 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: { 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: "labels": { 2026-02-19T13:35:59.733 INFO:tasks.workunit.client.0.trial144.stdout: "level": "shallow", 2026-02-19T13:35:59.734 INFO:tasks.workunit.client.0.trial144.stdout: "pooltype": "replicated" 2026-02-19T13:35:59.734 INFO:tasks.workunit.client.0.trial144.stdout: }, 2026-02-19T13:35:59.734 INFO:tasks.workunit.client.0.trial144.stdout: "counters": { 2026-02-19T13:35:59.734 INFO:tasks.workunit.client.0.trial144.stdout: "preemptions": 0, 2026-02-19T13:35:59.734 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_selected": 0, 2026-02-19T13:35:59.734 INFO:tasks.workunit.client.0.trial144.stdout: "chunk_busy": 0, 2026-02-19T13:35:59.734 INFO:tasks.workunit.client.0.trial144.stdout: "locked_object": 0, 2026-02-19T13:35:59.734 INFO:tasks.workunit.client.0.trial144.stdout: "write_blocked_by_scrub": 0 2026-02-19T13:35:59.734 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.734 INFO:tasks.workunit.client.0.trial144.stdout: } 2026-02-19T13:35:59.734 INFO:tasks.workunit.client.0.trial144.stdout: ] 2026-02-19T13:35:59.734 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:35:59.734 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Teardown Test TEST_scrub_test ------------------ 2026-02-19T13:35:59.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_scrub_test ------------------' 2026-02-19T13:35:59.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-19T13:35:59.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:35:59.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:35:59.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:35:59.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:35:59.735 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:35:59.735 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:35:59.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:35:59.736 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:36:00.042 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:36:00.042 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:36:00.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:36:00.043 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:36:00.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:36:00.045 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:36:00.045 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:36:00.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:36:00.046 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:36:00.047 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:36:00.047 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:36:00.048 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:36:00.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:36:00.049 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:36:00.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:36:00.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:00.061 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:36:00.061 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:36:00.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:36:00.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:36:00.063 INFO:tasks.workunit.client.0.trial144.stdout:-------------- Complete Test TEST_scrub_test ------------------ 2026-02-19T13:36:00.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_scrub_test ------------------' 2026-02-19T13:36:00.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-19T13:36:00.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/osd-scrub-test 0 2026-02-19T13:36:00.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-scrub-test 2026-02-19T13:36:00.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-19T13:36:00.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-19T13:36:00.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:36:00.065 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:36:00.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:36:00.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:36:00.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:36:00.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:36:00.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:36:00.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:36:00.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:36:00.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:36:00.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:36:00.068 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:36:00.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:36:00.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:36:00.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:36:00.070 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:36:00.071 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:36:00.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-19T13:36:00.072 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-scrub-test 2026-02-19T13:36:00.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:36:00.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:00.074 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.511299 2026-02-19T13:36:00.074 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.511299 2026-02-19T13:36:00.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:36:00.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:36:00.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-19T13:36:00.076 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-19T13:36:00.077 DEBUG:teuthology.orchestra.run.trial144:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-19T13:36:00.126 INFO:tasks.workunit:Running workunit scrub/osd-unexpected-clone.sh... 2026-02-19T13:36:00.127 DEBUG:teuthology.orchestra.run.trial144:workunit test scrub/osd-unexpected-clone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=624b9a7bcb2064ce17206482df6a1da007ff7675 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh 2026-02-19T13:36:00.179 INFO:tasks.workunit.client.0.trial144.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-19T13:36:00.182 INFO:tasks.workunit.client.0.trial144.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-19T13:36:00.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-19T13:36:00.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-19T13:36:00.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-19T13:36:00.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-19T13:36:00.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-19T13:36:00.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-19T13:36:00.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-19T13:36:00.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-19T13:36:00.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/osd-unexpected-clone 2026-02-19T13:36:00.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:22: run: local dir=td/osd-unexpected-clone 2026-02-19T13:36:00.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:23: run: shift 2026-02-19T13:36:00.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:25: run: export CEPH_MON=127.0.0.1:7144 2026-02-19T13:36:00.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:25: run: CEPH_MON=127.0.0.1:7144 2026-02-19T13:36:00.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:26: run: export CEPH_ARGS 2026-02-19T13:36:00.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:27: run: uuidgen 2026-02-19T13:36:00.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:27: run: CEPH_ARGS+='--fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none ' 2026-02-19T13:36:00.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7144 ' 2026-02-19T13:36:00.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:30: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-19T13:36:00.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:31: run: set 2026-02-19T13:36:00.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-19T13:36:00.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:31: run: local funcs=TEST_recover_unexpected 2026-02-19T13:36:00.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:32: run: for func in $funcs 2026-02-19T13:36:00.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:33: run: setup td/osd-unexpected-clone 2026-02-19T13:36:00.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-unexpected-clone 2026-02-19T13:36:00.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-unexpected-clone 2026-02-19T13:36:00.187 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-unexpected-clone 2026-02-19T13:36:00.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:36:00.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-unexpected-clone KILL 2026-02-19T13:36:00.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:36:00.188 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:36:00.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:36:00.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:36:00.188 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:36:00.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:36:00.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:36:00.189 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:36:00.189 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:36:00.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:36:00.191 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:36:00.191 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:36:00.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:36:00.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:36:00.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:36:00.192 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:36:00.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:36:00.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:36:00.194 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-unexpected-clone 2026-02-19T13:36:00.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:36:00.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:00.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:36:00.196 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.658971 2026-02-19T13:36:00.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:36:00.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:36:00.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-unexpected-clone 2026-02-19T13:36:00.199 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:36:00.200 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:00.200 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:36:00.200 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.658971 2026-02-19T13:36:00.201 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:36:00.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-19T13:36:00.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-19T13:36:00.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:36:00.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-unexpected-clone 1' TERM HUP INT 2026-02-19T13:36:00.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:34: run: TEST_recover_unexpected td/osd-unexpected-clone 2026-02-19T13:36:00.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:40: TEST_recover_unexpected: local dir=td/osd-unexpected-clone 2026-02-19T13:36:00.203 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:42: TEST_recover_unexpected: run_mon td/osd-unexpected-clone a 2026-02-19T13:36:00.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-unexpected-clone 2026-02-19T13:36:00.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:36:00.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:36:00.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:36:00.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-unexpected-clone/a 2026-02-19T13:36:00.204 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-unexpected-clone/a --run-dir=td/osd-unexpected-clone 2026-02-19T13:36:00.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:36:00.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:36:00.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:36:00.222 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:36:00.222 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:00.222 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:36:00.222 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:00.223 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-unexpected-clone/a '--log-file=td/osd-unexpected-clone/$name.log' '--admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' --mon-cluster-log-file=td/osd-unexpected-clone/log --run-dir=td/osd-unexpected-clone '--pid-file=td/osd-unexpected-clone/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:36:00.245 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:36:00.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:36:00.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:36:00.247 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:36:00.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:36:00.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:36:00.248 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:36:00.248 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:36:00.248 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:36:00.248 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:36:00.248 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:00.248 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:36:00.248 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.658971/ceph-mon.a.asok 2026-02-19T13:36:00.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:36:00.248 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.658971/ceph-mon.a.asok config get fsid 2026-02-19T13:36:00.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:36:00.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:36:00.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:36:00.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:36:00.302 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:36:00.303 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:36:00.303 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:36:00.303 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:36:00.303 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:36:00.303 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:00.303 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:36:00.303 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.658971/ceph-mon.a.asok 2026-02-19T13:36:00.303 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:36:00.304 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.658971/ceph-mon.a.asok config get mon_host 2026-02-19T13:36:00.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:43: TEST_recover_unexpected: run_mgr td/osd-unexpected-clone x 2026-02-19T13:36:00.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-unexpected-clone 2026-02-19T13:36:00.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:36:00.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:36:00.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:36:00.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-unexpected-clone/x 2026-02-19T13:36:00.357 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:36:00.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:36:00.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:36:00.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:36:00.450 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:36:00.450 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:00.451 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:36:00.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:00.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:36:00.451 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-unexpected-clone/x '--log-file=td/osd-unexpected-clone/$name.log' '--admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' --run-dir=td/osd-unexpected-clone '--pid-file=td/osd-unexpected-clone/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:36:00.468 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:44: TEST_recover_unexpected: run_osd td/osd-unexpected-clone 0 2026-02-19T13:36:00.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-unexpected-clone 2026-02-19T13:36:00.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:36:00.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:36:00.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:36:00.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-unexpected-clone/0 2026-02-19T13:36:00.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-19T13:36:00.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:36:00.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:36:00.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:36:00.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/0' 2026-02-19T13:36:00.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/0/journal' 2026-02-19T13:36:00.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:36:00.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:36:00.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-19T13:36:00.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:36:00.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:36:00.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:36:00.470 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:36:00.470 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:00.470 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:36:00.470 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:00.470 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:00.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:36:00.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:36:00.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:36:00.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-19T13:36:00.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-19T13:36:00.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:36:00.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:36:00.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:36:00.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:36:00.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:36:00.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:36:00.471 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-unexpected-clone/0 2026-02-19T13:36:00.472 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:36:00.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=65b78289-0ce8-48ef-8613-38a158797018 2026-02-19T13:36:00.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 65b78289-0ce8-48ef-8613-38a158797018' 2026-02-19T13:36:00.473 INFO:tasks.workunit.client.0.trial144.stdout:add osd0 65b78289-0ce8-48ef-8613-38a158797018 2026-02-19T13:36:00.473 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:36:00.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDAEZdpV7G1HBAAVx65+E7Y//jlrCfyARgzGQ== 2026-02-19T13:36:00.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDAEZdpV7G1HBAAVx65+E7Y//jlrCfyARgzGQ=="}' 2026-02-19T13:36:00.481 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 65b78289-0ce8-48ef-8613-38a158797018 -i td/osd-unexpected-clone/0/new.json 2026-02-19T13:36:00.570 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:36:00.578 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-unexpected-clone/0/new.json 2026-02-19T13:36:00.579 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/0 --osd-journal=td/osd-unexpected-clone/0/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDAEZdpV7G1HBAAVx65+E7Y//jlrCfyARgzGQ== --osd-uuid 65b78289-0ce8-48ef-8613-38a158797018 2026-02-19T13:36:00.595 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:00.595+0000 7f7c75c9ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:00.596 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:00.595+0000 7f7c75c9ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:00.597 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:00.595+0000 7f7c75c9ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:00.598 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:00.595+0000 7f7c75c9ba40 -1 bdev(0x5633ce414c00 td/osd-unexpected-clone/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:36:00.598 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:00.595+0000 7f7c75c9ba40 -1 bluestore(td/osd-unexpected-clone/0) _read_fsid unparsable uuid 2026-02-19T13:36:01.333 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-unexpected-clone/0/keyring 2026-02-19T13:36:01.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:36:01.334 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:36:01.335 INFO:tasks.workunit.client.0.trial144.stdout:adding osd0 key to auth repository 2026-02-19T13:36:01.335 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-unexpected-clone/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:36:01.533 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:36:01.533 INFO:tasks.workunit.client.0.trial144.stdout:start osd.0 2026-02-19T13:36:01.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/0 --osd-journal=td/osd-unexpected-clone/0/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:36:01.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:36:01.534 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:36:01.535 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:36:01.556 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:01.555+0000 7f9b2f40fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:01.557 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:01.555+0000 7f9b2f40fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:01.558 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:01.559+0000 7f9b2f40fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:01.688 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:36:01.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:36:01.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:36:01.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:36:01.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:36:01.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:01.689 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:36:01.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:36:01.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:01.690 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:36:01.851 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:36:01.924 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:01.923+0000 7f9b2f40fa40 -1 Falling back to public interface 2026-02-19T13:36:02.452 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:02.451+0000 7f9b2f40fa40 -1 osd.0 0 log_to_monitors true 2026-02-19T13:36:02.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:36:02.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:02.852 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:36:02.853 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:36:02.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:02.853 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:36:03.016 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:36:04.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:36:04.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:04.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:36:04.018 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:36:04.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:04.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:36:04.176 INFO:tasks.workunit.client.0.trial144.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/412872967,v1:127.0.0.1:6803/412872967] [v2:127.0.0.1:6804/412872967,v1:127.0.0.1:6805/412872967] exists,up 65b78289-0ce8-48ef-8613-38a158797018 2026-02-19T13:36:04.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:36:04.179 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:36:04.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:36:04.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:45: TEST_recover_unexpected: run_osd td/osd-unexpected-clone 1 2026-02-19T13:36:04.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-unexpected-clone 2026-02-19T13:36:04.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:36:04.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:36:04.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:36:04.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-unexpected-clone/1 2026-02-19T13:36:04.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-19T13:36:04.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:36:04.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:36:04.180 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:36:04.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/1' 2026-02-19T13:36:04.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/1/journal' 2026-02-19T13:36:04.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:36:04.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:36:04.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-19T13:36:04.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:36:04.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:36:04.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:36:04.181 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:36:04.181 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:04.181 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:36:04.181 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:04.181 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:04.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:36:04.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:36:04.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:36:04.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-19T13:36:04.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-19T13:36:04.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:36:04.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:36:04.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:36:04.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:36:04.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:36:04.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:36:04.182 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-unexpected-clone/1 2026-02-19T13:36:04.183 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:36:04.183 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2e5eb394-f393-4233-9bf8-11dc7820af74 2026-02-19T13:36:04.183 INFO:tasks.workunit.client.0.trial144.stdout:add osd1 2e5eb394-f393-4233-9bf8-11dc7820af74 2026-02-19T13:36:04.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 2e5eb394-f393-4233-9bf8-11dc7820af74' 2026-02-19T13:36:04.184 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:36:04.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDEEZdpP5+CCxAAZ+BofFfIxMlwC1VjbFqDmQ== 2026-02-19T13:36:04.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDEEZdpP5+CCxAAZ+BofFfIxMlwC1VjbFqDmQ=="}' 2026-02-19T13:36:04.192 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2e5eb394-f393-4233-9bf8-11dc7820af74 -i td/osd-unexpected-clone/1/new.json 2026-02-19T13:36:04.355 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:36:04.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-unexpected-clone/1/new.json 2026-02-19T13:36:04.365 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/1 --osd-journal=td/osd-unexpected-clone/1/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDEEZdpP5+CCxAAZ+BofFfIxMlwC1VjbFqDmQ== --osd-uuid 2e5eb394-f393-4233-9bf8-11dc7820af74 2026-02-19T13:36:04.382 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:04.379+0000 7fb20d48aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:04.384 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:04.383+0000 7fb20d48aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:04.385 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:04.383+0000 7fb20d48aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:04.385 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:04.383+0000 7fb20d48aa40 -1 bdev(0x55db23bacc00 td/osd-unexpected-clone/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:36:04.385 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:04.383+0000 7fb20d48aa40 -1 bluestore(td/osd-unexpected-clone/1) _read_fsid unparsable uuid 2026-02-19T13:36:05.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-unexpected-clone/1/keyring 2026-02-19T13:36:05.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:36:05.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:36:05.027 INFO:tasks.workunit.client.0.trial144.stdout:adding osd1 key to auth repository 2026-02-19T13:36:05.027 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-unexpected-clone/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:36:05.228 INFO:tasks.workunit.client.0.trial144.stdout:start osd.1 2026-02-19T13:36:05.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:36:05.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/1 --osd-journal=td/osd-unexpected-clone/1/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:36:05.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:36:05.229 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:36:05.230 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:36:05.251 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:05.251+0000 7f275026fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:05.253 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:05.251+0000 7f275026fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:05.254 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:05.251+0000 7f275026fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:05.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:36:05.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:36:05.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:36:05.385 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:36:05.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:36:05.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:05.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:36:05.386 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:36:05.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:05.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:36:05.546 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:36:05.612 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:05.611+0000 7f275026fa40 -1 Falling back to public interface 2026-02-19T13:36:06.168 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:06.167+0000 7f275026fa40 -1 osd.1 0 log_to_monitors true 2026-02-19T13:36:06.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:36:06.547 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:06.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:36:06.548 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:36:06.548 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:06.549 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:36:06.709 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:36:07.710 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:36:07.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:07.711 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:36:07.711 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:36:07.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:07.712 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:36:07.865 INFO:tasks.workunit.client.0.trial144.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3356510000,v1:127.0.0.1:6811/3356510000] [v2:127.0.0.1:6812/3356510000,v1:127.0.0.1:6813/3356510000] exists,up 2e5eb394-f393-4233-9bf8-11dc7820af74 2026-02-19T13:36:07.868 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:36:07.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:36:07.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:36:07.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:46: TEST_recover_unexpected: run_osd td/osd-unexpected-clone 2 2026-02-19T13:36:07.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-unexpected-clone 2026-02-19T13:36:07.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:36:07.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:36:07.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:36:07.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-unexpected-clone/2 2026-02-19T13:36:07.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-19T13:36:07.869 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:36:07.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:36:07.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:36:07.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/2' 2026-02-19T13:36:07.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/2/journal' 2026-02-19T13:36:07.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:36:07.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:36:07.870 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-19T13:36:07.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:36:07.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:36:07.870 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:36:07.870 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:36:07.870 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:07.870 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:36:07.871 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:07.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:07.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:36:07.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:36:07.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:36:07.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-19T13:36:07.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-19T13:36:07.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:36:07.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:36:07.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:36:07.871 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:36:07.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:36:07.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:36:07.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-unexpected-clone/2 2026-02-19T13:36:07.872 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:36:07.872 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=662b94a5-da47-45b4-8613-8a7d390dc054 2026-02-19T13:36:07.873 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 662b94a5-da47-45b4-8613-8a7d390dc054' 2026-02-19T13:36:07.873 INFO:tasks.workunit.client.0.trial144.stdout:add osd2 662b94a5-da47-45b4-8613-8a7d390dc054 2026-02-19T13:36:07.873 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:36:07.881 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDHEZdp7qmbNBAACGwdaXxUkgXQ+C5N11IN2w== 2026-02-19T13:36:07.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDHEZdp7qmbNBAACGwdaXxUkgXQ+C5N11IN2w=="}' 2026-02-19T13:36:07.882 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 662b94a5-da47-45b4-8613-8a7d390dc054 -i td/osd-unexpected-clone/2/new.json 2026-02-19T13:36:08.044 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:36:08.054 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-unexpected-clone/2/new.json 2026-02-19T13:36:08.055 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/2 --osd-journal=td/osd-unexpected-clone/2/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDHEZdp7qmbNBAACGwdaXxUkgXQ+C5N11IN2w== --osd-uuid 662b94a5-da47-45b4-8613-8a7d390dc054 2026-02-19T13:36:08.071 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:08.071+0000 7fe653aafa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:08.072 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:08.071+0000 7fe653aafa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:08.074 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:08.071+0000 7fe653aafa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:08.074 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:08.071+0000 7fe653aafa40 -1 bdev(0x5617fc3b4c00 td/osd-unexpected-clone/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:36:08.074 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:08.071+0000 7fe653aafa40 -1 bluestore(td/osd-unexpected-clone/2) _read_fsid unparsable uuid 2026-02-19T13:36:08.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-unexpected-clone/2/keyring 2026-02-19T13:36:08.717 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:36:08.718 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:36:08.718 INFO:tasks.workunit.client.0.trial144.stdout:adding osd2 key to auth repository 2026-02-19T13:36:08.719 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-unexpected-clone/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:36:08.923 INFO:tasks.workunit.client.0.trial144.stdout:start osd.2 2026-02-19T13:36:08.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:36:08.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/2 --osd-journal=td/osd-unexpected-clone/2/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:36:08.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:36:08.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:36:08.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:36:08.945 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:08.943+0000 7fc3179bba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:08.947 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:08.947+0000 7fc3179bba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:08.948 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:08.947+0000 7fc3179bba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:09.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:36:09.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:36:09.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:36:09.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:36:09.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:36:09.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:09.079 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:36:09.080 INFO:tasks.workunit.client.0.trial144.stdout:0 2026-02-19T13:36:09.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:09.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:36:09.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:36:09.344 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:09.343+0000 7fc3179bba40 -1 Falling back to public interface 2026-02-19T13:36:09.880 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:09.879+0000 7fc3179bba40 -1 osd.2 0 log_to_monitors true 2026-02-19T13:36:10.237 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:36:10.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:10.238 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:36:10.238 INFO:tasks.workunit.client.0.trial144.stdout:1 2026-02-19T13:36:10.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:10.239 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:36:10.395 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:36:11.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:36:11.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:11.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-19T13:36:11.396 INFO:tasks.workunit.client.0.trial144.stdout:2 2026-02-19T13:36:11.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:11.396 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:36:11.551 INFO:tasks.workunit.client.0.trial144.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1792440969,v1:127.0.0.1:6819/1792440969] [v2:127.0.0.1:6820/1792440969,v1:127.0.0.1:6821/1792440969] exists,up 662b94a5-da47-45b4-8613-8a7d390dc054 2026-02-19T13:36:11.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:36:11.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:36:11.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:36:11.552 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:48: TEST_recover_unexpected: ceph osd pool create foo 1 2026-02-19T13:36:11.722 INFO:tasks.workunit.client.0.trial144.stderr:pool 'foo' created 2026-02-19T13:36:11.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:49: TEST_recover_unexpected: rados -p foo put foo /etc/passwd 2026-02-19T13:36:12.867 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:50: TEST_recover_unexpected: rados -p foo mksnap snap 2026-02-19T13:36:12.914 INFO:tasks.workunit.client.0.trial144.stdout:created pool foo snap snap 2026-02-19T13:36:12.916 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:51: TEST_recover_unexpected: rados -p foo put foo /etc/group 2026-02-19T13:36:12.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:53: TEST_recover_unexpected: wait_for_clean 2026-02-19T13:36:12.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:36:12.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:36:12.940 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:36:12.941 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:36:12.941 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:36:12.942 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:36:12.942 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:36:12.942 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:36:12.942 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:36:13.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:36:13.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:36:13.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:36:13.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:36:13.053 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:36:13.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:36:13.210 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:36:13.210 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:36:13.210 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:36:13.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:36:13.211 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:13.211 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:36:13.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-19T13:36:13.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-19T13:36:13.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-19T13:36:13.282 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:13.282 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:36:13.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672964 2026-02-19T13:36:13.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672964 2026-02-19T13:36:13.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-42949672964' 2026-02-19T13:36:13.356 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:13.356 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:36:13.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542147 2026-02-19T13:36:13.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542147 2026-02-19T13:36:13.430 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-42949672964 2-60129542147' 2026-02-19T13:36:13.431 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:13.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-19T13:36:13.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:13.432 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:36:13.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-19T13:36:13.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:13.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-19T13:36:13.434 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-19T13:36:13.434 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.0 seq 21474836484 2026-02-19T13:36:13.435 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:36:13.599 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-19T13:36:13.600 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:36:14.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:36:14.601 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:36:14.762 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-19T13:36:14.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:36:15.764 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:36:15.764 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:36:15.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836484 2026-02-19T13:36:15.922 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:15.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672964 2026-02-19T13:36:15.923 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:15.924 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:36:15.924 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672964 2026-02-19T13:36:15.925 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:15.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672964 2026-02-19T13:36:15.926 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.1 seq 42949672964 2026-02-19T13:36:15.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-19T13:36:15.926 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:36:16.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-19T13:36:16.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:16.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542147 2026-02-19T13:36:16.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:16.090 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:36:16.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542147 2026-02-19T13:36:16.091 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:16.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542147 2026-02-19T13:36:16.092 INFO:tasks.workunit.client.0.trial144.stdout:waiting osd.2 seq 60129542147 2026-02-19T13:36:16.092 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-19T13:36:16.093 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:36:16.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-19T13:36:16.260 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:36:16.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:16.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:16.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:36:16.450 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:36:16.450 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:36:16.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:36:16.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:36:16.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:36:16.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:36:16.451 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:36:16.601 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:36:16.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:36:16.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:16.602 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:16.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:36:16.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:36:16.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:36:16.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:55: TEST_recover_unexpected: get_primary foo foo 2026-02-19T13:36:16.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=foo 2026-02-19T13:36:16.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=foo 2026-02-19T13:36:16.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map foo foo 2026-02-19T13:36:16.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:36:16.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:55: TEST_recover_unexpected: local osd=1 2026-02-19T13:36:16.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:57: TEST_recover_unexpected: objectstore_tool td/osd-unexpected-clone 1 --op list foo 2026-02-19T13:36:16.963 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:57: TEST_recover_unexpected: grep snapid.:1 2026-02-19T13:36:16.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-unexpected-clone 2026-02-19T13:36:16.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:36:16.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:36:16.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:36:16.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-unexpected-clone 1 --op list foo 2026-02-19T13:36:16.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-unexpected-clone 2026-02-19T13:36:16.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:36:16.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:36:16.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:36:16.964 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-unexpected-clone TERM osd.1 2026-02-19T13:36:16.965 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:36:16.965 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:36:16.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:36:16.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:36:16.965 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:36:17.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:36:17.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-unexpected-clone 1 --op list foo 2026-02-19T13:36:17.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-unexpected-clone 2026-02-19T13:36:17.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:36:17.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:36:17.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:36:17.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-unexpected-clone/1 2026-02-19T13:36:17.271 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-unexpected-clone/1 --op list foo 2026-02-19T13:36:17.428 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-unexpected-clone 1 2026-02-19T13:36:17.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-unexpected-clone 2026-02-19T13:36:17.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:36:17.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:36:17.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:36:17.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-unexpected-clone/1 2026-02-19T13:36:17.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-19T13:36:17.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:36:17.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:36:17.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:36:17.429 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/1' 2026-02-19T13:36:17.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/1/journal' 2026-02-19T13:36:17.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:36:17.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:36:17.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-19T13:36:17.430 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:36:17.430 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:36:17.430 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:36:17.430 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:36:17.430 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:17.430 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:36:17.430 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:17.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:17.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:36:17.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-19T13:36:17.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-19T13:36:17.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:36:17.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:36:17.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:36:17.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:36:17.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:36:17.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:36:17.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-unexpected-clone/1 2026-02-19T13:36:17.432 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:36:17.433 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:36:17.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/1 --osd-journal=td/osd-unexpected-clone/1/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:36:17.433 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-unexpected-clone/1/whoami 2026-02-19T13:36:17.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:36:17.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:36:17.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:36:17.433 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:36:17.448 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:17.447+0000 7f46a1221a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:17.449 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:17.447+0000 7f46a1221a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:17.450 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:17.451+0000 7f46a1221a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:17.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:36:17.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:36:17.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:36:17.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:36:17.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:36:17.593 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:17.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:36:17.594 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:36:17.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:17.594 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:36:17.747 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:36:17.824 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:17.823+0000 7f46a1221a40 -1 Falling back to public interface 2026-02-19T13:36:18.461 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:18.459+0000 7f46a1221a40 -1 osd.1 17 log_to_monitors true 2026-02-19T13:36:18.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:36:18.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:18.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:36:18.749 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:36:18.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:18.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:36:18.905 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 21 up_thru 21 down_at 18 last_clean_interval [10,17) [v2:127.0.0.1:6810/1040842285,v1:127.0.0.1:6811/1040842285] [v2:127.0.0.1:6812/1040842285,v1:127.0.0.1:6813/1040842285] exists,up 2e5eb394-f393-4233-9bf8-11dc7820af74 2026-02-19T13:36:18.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:36:18.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:36:18.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:36:18.905 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:36:18.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:36:18.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:36:18.906 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:36:18.906 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:36:18.907 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:36:18.907 INFO:tasks.workunit.client.0.trial144.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:36:18.907 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:36:18.907 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:36:18.907 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:36:19.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:36:19.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:36:19.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:36:19.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:36:19.007 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:36:19.008 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:36:19.170 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:36:19.171 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:36:19.171 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:36:19.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:36:19.171 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:19.171 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:36:19.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-19T13:36:19.241 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-19T13:36:19.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-19T13:36:19.242 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:19.242 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:36:19.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313218 2026-02-19T13:36:19.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313218 2026-02-19T13:36:19.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-90194313218' 2026-02-19T13:36:19.315 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:19.316 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:36:19.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542149 2026-02-19T13:36:19.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542149 2026-02-19T13:36:19.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-90194313218 2-60129542149' 2026-02-19T13:36:19.386 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:19.387 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-19T13:36:19.387 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:19.387 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:36:19.388 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-19T13:36:19.388 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:19.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-19T13:36:19.389 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-19T13:36:19.389 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836487 2026-02-19T13:36:19.390 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:36:19.554 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T13:36:19.554 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:36:20.555 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:36:20.555 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:36:20.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-19T13:36:20.717 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:36:21.718 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:36:21.719 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:36:21.883 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-19T13:36:21.883 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:21.884 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-90194313218 2026-02-19T13:36:21.884 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:21.885 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:36:21.885 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-90194313218 2026-02-19T13:36:21.885 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:21.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313218 2026-02-19T13:36:21.887 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 90194313218' 2026-02-19T13:36:21.887 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 90194313218 2026-02-19T13:36:21.887 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:36:22.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313218 -lt 90194313218 2026-02-19T13:36:22.045 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:22.045 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542149 2026-02-19T13:36:22.045 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:22.046 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:36:22.047 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542149 2026-02-19T13:36:22.047 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:22.048 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542149 2026-02-19T13:36:22.049 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542149' 2026-02-19T13:36:22.049 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542149 2026-02-19T13:36:22.049 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:36:22.219 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542149 -lt 60129542149 2026-02-19T13:36:22.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:36:22.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:22.220 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:22.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:36:22.415 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:36:22.415 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:36:22.415 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:36:22.415 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:36:22.415 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:36:22.416 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:36:22.416 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:36:22.569 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:36:22.569 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:36:22.570 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:22.570 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:22.755 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:36:22.755 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:36:22.755 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:36:22.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:57: TEST_recover_unexpected: JSON='["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:36:22.756 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:58: TEST_recover_unexpected: echo 'JSON is ["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-19T13:36:22.756 INFO:tasks.workunit.client.0.trial144.stdout:JSON is ["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}] 2026-02-19T13:36:22.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:59: TEST_recover_unexpected: rm -f td/osd-unexpected-clone/_ td/osd-unexpected-clone/data 2026-02-19T13:36:22.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:60: TEST_recover_unexpected: objectstore_tool td/osd-unexpected-clone 1 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-attr _ 2026-02-19T13:36:22.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-unexpected-clone 2026-02-19T13:36:22.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:36:22.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:36:22.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:36:22.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-unexpected-clone 1 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-attr _ 2026-02-19T13:36:22.758 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-unexpected-clone 2026-02-19T13:36:22.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:36:22.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:36:22.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:36:22.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-unexpected-clone TERM osd.1 2026-02-19T13:36:22.759 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:36:22.759 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:36:22.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:36:22.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:36:22.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:36:23.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:36:23.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-unexpected-clone 1 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-attr _ 2026-02-19T13:36:23.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-unexpected-clone 2026-02-19T13:36:23.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:36:23.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:36:23.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:36:23.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-unexpected-clone/1 2026-02-19T13:36:23.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-unexpected-clone/1 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-attr _ 2026-02-19T13:36:23.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-unexpected-clone 1 2026-02-19T13:36:23.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-unexpected-clone 2026-02-19T13:36:23.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:36:23.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:36:23.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:36:23.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-unexpected-clone/1 2026-02-19T13:36:23.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-19T13:36:23.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:36:23.267 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:36:23.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:36:23.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/1' 2026-02-19T13:36:23.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/1/journal' 2026-02-19T13:36:23.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:36:23.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:36:23.268 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-19T13:36:23.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:36:23.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:36:23.268 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:36:23.268 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:36:23.268 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:23.268 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:36:23.269 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:23.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:23.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:36:23.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-19T13:36:23.269 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-19T13:36:23.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:36:23.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:36:23.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:36:23.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:36:23.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:36:23.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:36:23.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-unexpected-clone/1 2026-02-19T13:36:23.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:36:23.270 INFO:tasks.workunit.client.0.trial144.stderr:start osd.1 2026-02-19T13:36:23.270 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/1 --osd-journal=td/osd-unexpected-clone/1/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:36:23.270 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-unexpected-clone/1/whoami 2026-02-19T13:36:23.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:36:23.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:36:23.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:36:23.271 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:36:23.286 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:23.282+0000 7fcd1f86ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:23.288 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:23.286+0000 7fcd1f86ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:23.289 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:23.286+0000 7fcd1f86ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:23.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:36:23.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:36:23.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:36:23.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:36:23.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:36:23.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:23.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:36:23.427 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:36:23.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:23.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:36:23.589 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:36:23.672 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:23.670+0000 7fcd1f86ba40 -1 Falling back to public interface 2026-02-19T13:36:24.269 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:24.266+0000 7fcd1f86ba40 -1 osd.1 23 log_to_monitors true 2026-02-19T13:36:24.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:36:24.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:24.590 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:36:24.590 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:36:24.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:24.591 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:36:24.748 INFO:tasks.workunit.client.0.trial144.stderr:osd.1 up in weight 1 up_from 27 up_thru 27 down_at 24 last_clean_interval [21,23) [v2:127.0.0.1:6810/4132723380,v1:127.0.0.1:6811/4132723380] [v2:127.0.0.1:6812/4132723380,v1:127.0.0.1:6813/4132723380] exists,up 2e5eb394-f393-4233-9bf8-11dc7820af74 2026-02-19T13:36:24.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:36:24.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:36:24.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:36:24.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:36:24.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:36:24.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:36:24.749 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:36:24.749 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:36:24.750 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:36:24.750 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:36:24.750 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:36:24.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:36:24.751 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:36:24.849 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:36:24.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:36:24.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:36:24.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:36:24.850 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:36:24.850 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:36:25.017 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:36:25.018 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:36:25.018 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:36:25.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:36:25.018 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:25.018 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:36:25.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836489 2026-02-19T13:36:25.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836489 2026-02-19T13:36:25.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489' 2026-02-19T13:36:25.089 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:25.089 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:36:25.160 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116994 2026-02-19T13:36:25.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116994 2026-02-19T13:36:25.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-115964116994' 2026-02-19T13:36:25.161 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:25.161 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:36:25.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542151 2026-02-19T13:36:25.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542151 2026-02-19T13:36:25.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-115964116994 2-60129542151' 2026-02-19T13:36:25.232 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:25.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836489 2026-02-19T13:36:25.233 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:25.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:36:25.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836489 2026-02-19T13:36:25.234 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:25.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836489 2026-02-19T13:36:25.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-19T13:36:25.236 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836489 2026-02-19T13:36:25.236 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:36:25.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-19T13:36:25.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:36:26.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:36:26.406 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:36:26.565 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-19T13:36:26.566 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:36:27.567 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:36:27.567 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:36:27.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-19T13:36:27.722 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:27.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-115964116994 2026-02-19T13:36:27.723 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:27.724 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:36:27.724 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-115964116994 2026-02-19T13:36:27.725 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:27.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964116994 2026-02-19T13:36:27.726 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 115964116994' 2026-02-19T13:36:27.726 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 115964116994 2026-02-19T13:36:27.726 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:36:27.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116994 -lt 115964116994 2026-02-19T13:36:27.897 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:27.897 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542151 2026-02-19T13:36:27.898 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:27.898 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:36:27.899 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542151 2026-02-19T13:36:27.900 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:27.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542151 2026-02-19T13:36:27.901 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542151' 2026-02-19T13:36:27.901 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 60129542151 2026-02-19T13:36:27.901 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:36:28.058 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542152 -lt 60129542151 2026-02-19T13:36:28.058 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:36:28.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:28.059 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:28.248 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:36:28.249 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:36:28.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:36:28.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:36:28.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:36:28.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:36:28.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:36:28.250 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:36:28.403 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:36:28.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:36:28.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:28.404 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:28.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:36:28.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:36:28.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:36:28.595 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:61: TEST_recover_unexpected: objectstore_tool td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-bytes td/osd-unexpected-clone/data 2026-02-19T13:36:28.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-unexpected-clone 2026-02-19T13:36:28.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:36:28.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T13:36:28.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:36:28.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-bytes td/osd-unexpected-clone/data 2026-02-19T13:36:28.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-unexpected-clone 2026-02-19T13:36:28.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:36:28.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T13:36:28.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:36:28.596 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-unexpected-clone TERM osd.2 2026-02-19T13:36:28.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:36:28.596 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:36:28.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:36:28.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:36:28.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:36:28.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:36:28.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-bytes td/osd-unexpected-clone/data 2026-02-19T13:36:28.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-unexpected-clone 2026-02-19T13:36:28.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:36:28.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T13:36:28.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:36:28.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-unexpected-clone/2 2026-02-19T13:36:28.903 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-unexpected-clone/2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-bytes td/osd-unexpected-clone/data 2026-02-19T13:36:29.064 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-unexpected-clone 2 2026-02-19T13:36:29.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-unexpected-clone 2026-02-19T13:36:29.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:36:29.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:36:29.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:36:29.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-unexpected-clone/2 2026-02-19T13:36:29.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-19T13:36:29.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:36:29.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:36:29.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:36:29.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/2' 2026-02-19T13:36:29.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/2/journal' 2026-02-19T13:36:29.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:36:29.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:36:29.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-19T13:36:29.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:36:29.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:36:29.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:36:29.066 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:36:29.066 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:29.066 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:36:29.066 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:29.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:29.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:36:29.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-19T13:36:29.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-19T13:36:29.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:36:29.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:36:29.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:36:29.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:36:29.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:36:29.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:36:29.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-unexpected-clone/2 2026-02-19T13:36:29.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:36:29.069 INFO:tasks.workunit.client.0.trial144.stderr:start osd.2 2026-02-19T13:36:29.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/2 --osd-journal=td/osd-unexpected-clone/2/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:36:29.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-unexpected-clone/2/whoami 2026-02-19T13:36:29.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:36:29.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:36:29.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:36:29.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:36:29.089 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:29.086+0000 7efe053eca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:29.090 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:29.090+0000 7efe053eca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:29.091 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:29.090+0000 7efe053eca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:29.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:36:29.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:36:29.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:36:29.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:36:29.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:36:29.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:29.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:36:29.225 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:36:29.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:29.225 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:36:29.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:36:29.500 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:29.498+0000 7efe053eca40 -1 Falling back to public interface 2026-02-19T13:36:30.165 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:30.162+0000 7efe053eca40 -1 osd.2 29 log_to_monitors true 2026-02-19T13:36:30.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:36:30.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:30.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:36:30.383 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:36:30.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:30.383 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:36:30.542 INFO:tasks.workunit.client.0.trial144.stderr:osd.2 up in weight 1 up_from 32 up_thru 0 down_at 29 last_clean_interval [14,29) [v2:127.0.0.1:6818/2341786058,v1:127.0.0.1:6819/2341786058] [v2:127.0.0.1:6820/2341786058,v1:127.0.0.1:6821/2341786058] exists,up 662b94a5-da47-45b4-8613-8a7d390dc054 2026-02-19T13:36:30.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:36:30.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:36:30.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:36:30.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:36:30.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:36:30.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:36:30.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:36:30.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:36:30.543 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:36:30.543 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:36:30.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:36:30.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:36:30.544 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:36:30.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:36:30.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:36:30.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:36:30.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:36:30.641 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:36:30.641 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:36:30.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:36:30.804 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:36:30.804 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:36:30.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:36:30.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:30.805 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:36:30.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-19T13:36:30.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-19T13:36:30.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491' 2026-02-19T13:36:30.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:30.877 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:36:30.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116996 2026-02-19T13:36:30.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116996 2026-02-19T13:36:30.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-115964116996' 2026-02-19T13:36:30.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:30.947 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:36:31.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=137438953474 2026-02-19T13:36:31.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 137438953474 2026-02-19T13:36:31.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-115964116996 2-137438953474' 2026-02-19T13:36:31.021 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:31.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-19T13:36:31.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:31.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:36:31.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836491 2026-02-19T13:36:31.023 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:31.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836491 2026-02-19T13:36:31.025 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-19T13:36:31.025 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836491 2026-02-19T13:36:31.025 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:36:31.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-19T13:36:31.193 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:31.194 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-115964116996 2026-02-19T13:36:31.194 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:31.195 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:36:31.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-115964116996 2026-02-19T13:36:31.196 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:31.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964116996 2026-02-19T13:36:31.197 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 115964116996' 2026-02-19T13:36:31.197 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 115964116996 2026-02-19T13:36:31.197 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:36:31.363 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116995 -lt 115964116996 2026-02-19T13:36:31.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:36:32.364 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:36:32.365 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:36:32.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116995 -lt 115964116996 2026-02-19T13:36:32.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:36:33.524 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:36:33.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:36:33.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116996 -lt 115964116996 2026-02-19T13:36:33.685 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:33.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-137438953474 2026-02-19T13:36:33.686 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:33.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:36:33.688 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-137438953474 2026-02-19T13:36:33.688 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:33.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=137438953474 2026-02-19T13:36:33.689 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 137438953474' 2026-02-19T13:36:33.689 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 137438953474 2026-02-19T13:36:33.689 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:36:33.860 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 137438953474 -lt 137438953474 2026-02-19T13:36:33.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:36:33.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:33.861 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:34.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:36:34.052 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:36:34.052 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:36:34.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:36:34.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:36:34.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:36:34.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:36:34.053 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:36:34.208 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:36:34.208 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:36:34.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:34.209 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:34.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:36:34.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:36:34.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:36:34.401 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:63: TEST_recover_unexpected: rados -p foo rmsnap snap 2026-02-19T13:36:34.467 INFO:tasks.workunit.client.0.trial144.stdout:removed pool foo snap snap 2026-02-19T13:36:34.469 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:65: TEST_recover_unexpected: sleep 5 2026-02-19T13:36:39.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:67: TEST_recover_unexpected: objectstore_tool td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes td/osd-unexpected-clone/data 2026-02-19T13:36:39.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-unexpected-clone 2026-02-19T13:36:39.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:36:39.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T13:36:39.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:36:39.472 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes td/osd-unexpected-clone/data 2026-02-19T13:36:39.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-unexpected-clone 2026-02-19T13:36:39.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:36:39.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T13:36:39.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:36:39.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-unexpected-clone TERM osd.2 2026-02-19T13:36:39.473 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:36:39.473 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:36:39.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:36:39.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:36:39.473 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:36:39.782 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:36:39.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes td/osd-unexpected-clone/data 2026-02-19T13:36:39.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-unexpected-clone 2026-02-19T13:36:39.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:36:39.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T13:36:39.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:36:39.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-unexpected-clone/2 2026-02-19T13:36:39.783 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-unexpected-clone/2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes td/osd-unexpected-clone/data 2026-02-19T13:36:40.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-unexpected-clone 2 2026-02-19T13:36:40.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-unexpected-clone 2026-02-19T13:36:40.065 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:36:40.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:36:40.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:36:40.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-unexpected-clone/2 2026-02-19T13:36:40.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-19T13:36:40.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:36:40.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:36:40.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:36:40.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/2' 2026-02-19T13:36:40.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/2/journal' 2026-02-19T13:36:40.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:36:40.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:36:40.066 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-19T13:36:40.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:36:40.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:36:40.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:36:40.067 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:36:40.067 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:40.067 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:36:40.067 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:40.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:40.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:36:40.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-19T13:36:40.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-19T13:36:40.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:36:40.067 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:36:40.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:36:40.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:36:40.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:36:40.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:36:40.068 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-unexpected-clone/2 2026-02-19T13:36:40.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:36:40.069 INFO:tasks.workunit.client.0.trial144.stderr:start osd.2 2026-02-19T13:36:40.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/2 --osd-journal=td/osd-unexpected-clone/2/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:36:40.069 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-unexpected-clone/2/whoami 2026-02-19T13:36:40.069 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:36:40.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:36:40.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:36:40.070 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:36:40.083 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:40.082+0000 7fa01c283a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:40.084 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:40.082+0000 7fa01c283a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:40.085 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:40.082+0000 7fa01c283a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:40.226 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:36:40.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:36:40.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:36:40.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:36:40.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:36:40.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:40.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:36:40.227 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:36:40.227 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:40.228 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:36:40.384 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:36:40.476 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:40.474+0000 7fa01c283a40 -1 Falling back to public interface 2026-02-19T13:36:41.089 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:41.086+0000 7fa01c283a40 -1 osd.2 35 log_to_monitors true 2026-02-19T13:36:41.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:36:41.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:41.386 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:36:41.386 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:36:41.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:41.387 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:36:41.540 INFO:tasks.workunit.client.0.trial144.stderr:osd.2 up in weight 1 up_from 39 up_thru 0 down_at 36 last_clean_interval [32,35) [v2:127.0.0.1:6818/2275292889,v1:127.0.0.1:6819/2275292889] [v2:127.0.0.1:6820/2275292889,v1:127.0.0.1:6821/2275292889] exists,up 662b94a5-da47-45b4-8613-8a7d390dc054 2026-02-19T13:36:41.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:36:41.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:36:41.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:36:41.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:36:41.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:36:41.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:36:41.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:36:41.541 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:36:41.542 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:36:41.542 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:36:41.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:36:41.542 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:36:41.543 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:36:41.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:36:41.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:36:41.648 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:36:41.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:36:41.649 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:36:41.649 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:36:41.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:36:41.804 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:36:41.804 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:36:41.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:36:41.804 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:41.804 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:36:41.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836494 2026-02-19T13:36:41.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836494 2026-02-19T13:36:41.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836494' 2026-02-19T13:36:41.877 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:41.877 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:36:41.947 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116999 2026-02-19T13:36:41.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116999 2026-02-19T13:36:41.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836494 1-115964116999' 2026-02-19T13:36:41.948 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:41.948 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:36:42.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724546 2026-02-19T13:36:42.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724546 2026-02-19T13:36:42.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836494 1-115964116999 2-167503724546' 2026-02-19T13:36:42.022 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:42.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836494 2026-02-19T13:36:42.022 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:42.023 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:36:42.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836494 2026-02-19T13:36:42.024 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:42.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836494 2026-02-19T13:36:42.026 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836494' 2026-02-19T13:36:42.026 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836494 2026-02-19T13:36:42.026 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:36:42.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836493 -lt 21474836494 2026-02-19T13:36:42.184 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:36:43.185 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:36:43.186 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:36:43.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836494 -lt 21474836494 2026-02-19T13:36:43.345 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:43.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-115964116999 2026-02-19T13:36:43.346 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:43.347 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:36:43.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-115964116999 2026-02-19T13:36:43.348 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:43.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964116999 2026-02-19T13:36:43.349 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 115964116999' 2026-02-19T13:36:43.349 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 115964116999 2026-02-19T13:36:43.349 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:36:43.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116999 -lt 115964116999 2026-02-19T13:36:43.522 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:43.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-167503724546 2026-02-19T13:36:43.522 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:43.523 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:36:43.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-167503724546 2026-02-19T13:36:43.524 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:43.525 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724546 2026-02-19T13:36:43.526 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 167503724546' 2026-02-19T13:36:43.526 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 167503724546 2026-02-19T13:36:43.526 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:36:43.696 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724546 -lt 167503724546 2026-02-19T13:36:43.696 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:36:43.697 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:43.697 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:43.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:36:43.886 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:36:43.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:36:43.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:36:43.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:36:43.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:36:43.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:36:43.886 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:36:44.043 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:36:44.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:36:44.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:44.044 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:44.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:36:44.233 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:36:44.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:36:44.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:68: TEST_recover_unexpected: objectstore_tool td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-attr _ td/osd-unexpected-clone/_ 2026-02-19T13:36:44.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/osd-unexpected-clone 2026-02-19T13:36:44.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:36:44.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T13:36:44.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:36:44.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-attr _ td/osd-unexpected-clone/_ 2026-02-19T13:36:44.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/osd-unexpected-clone 2026-02-19T13:36:44.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:36:44.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T13:36:44.234 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:36:44.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/osd-unexpected-clone TERM osd.2 2026-02-19T13:36:44.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:36:44.235 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:36:44.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:36:44.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:36:44.235 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:36:44.541 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:36:44.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-attr _ td/osd-unexpected-clone/_ 2026-02-19T13:36:44.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/osd-unexpected-clone 2026-02-19T13:36:44.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:36:44.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T13:36:44.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:36:44.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/osd-unexpected-clone/2 2026-02-19T13:36:44.542 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-unexpected-clone/2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-attr _ td/osd-unexpected-clone/_ 2026-02-19T13:36:44.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/osd-unexpected-clone 2 2026-02-19T13:36:44.809 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-unexpected-clone 2026-02-19T13:36:44.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:36:44.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:36:44.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:36:44.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-unexpected-clone/2 2026-02-19T13:36:44.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-19T13:36:44.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:36:44.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:36:44.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:36:44.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/2' 2026-02-19T13:36:44.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/2/journal' 2026-02-19T13:36:44.810 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:36:44.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:36:44.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-19T13:36:44.811 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:36:44.811 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:36:44.811 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:36:44.811 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:36:44.811 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:36:44.811 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:36:44.811 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:44.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' 2026-02-19T13:36:44.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:36:44.811 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-19T13:36:44.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-19T13:36:44.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:36:44.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:36:44.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:36:44.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:36:44.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:36:44.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:36:44.812 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-unexpected-clone/2 2026-02-19T13:36:44.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:36:44.813 INFO:tasks.workunit.client.0.trial144.stderr:start osd.2 2026-02-19T13:36:44.813 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=53ab1e13-42ca-420f-8a5c-9319b1182dae --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/2 --osd-journal=td/osd-unexpected-clone/2/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.658971/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:36:44.813 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-unexpected-clone/2/whoami 2026-02-19T13:36:44.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:36:44.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:36:44.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:36:44.814 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:36:44.828 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:44.826+0000 7fda975cea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:44.830 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:44.826+0000 7fda975cea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:44.831 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:44.830+0000 7fda975cea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:36:44.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:36:44.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:36:44.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:36:44.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:36:44.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:36:44.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:44.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:36:44.970 INFO:tasks.workunit.client.0.trial144.stderr:0 2026-02-19T13:36:44.970 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:44.971 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:36:45.127 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:36:45.216 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:45.214+0000 7fda975cea40 -1 Falling back to public interface 2026-02-19T13:36:45.853 INFO:tasks.workunit.client.0.trial144.stderr:2026-02-19T13:36:45.850+0000 7fda975cea40 -1 osd.2 40 log_to_monitors true 2026-02-19T13:36:46.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:36:46.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:36:46.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:36:46.129 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:36:46.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:36:46.129 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:36:46.282 INFO:tasks.workunit.client.0.trial144.stderr:osd.2 up in weight 1 up_from 45 up_thru 0 down_at 41 last_clean_interval [39,40) [v2:127.0.0.1:6818/3538603586,v1:127.0.0.1:6819/3538603586] [v2:127.0.0.1:6820/3538603586,v1:127.0.0.1:6821/3538603586] exists,up 662b94a5-da47-45b4-8613-8a7d390dc054 2026-02-19T13:36:46.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:36:46.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:36:46.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:36:46.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:36:46.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:36:46.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:36:46.283 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:36:46.283 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:36:46.283 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:36:46.283 INFO:tasks.workunit.client.0.trial144.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:36:46.284 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:36:46.284 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:36:46.284 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:36:46.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:36:46.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:36:46.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:36:46.380 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:36:46.381 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:36:46.381 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:36:46.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:36:46.544 INFO:tasks.workunit.client.0.trial144.stderr:1 2026-02-19T13:36:46.544 INFO:tasks.workunit.client.0.trial144.stderr:2' 2026-02-19T13:36:46.544 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:36:46.545 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:46.545 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:36:46.617 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836496 2026-02-19T13:36:46.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836496 2026-02-19T13:36:46.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836496' 2026-02-19T13:36:46.618 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:46.618 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:36:46.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117001 2026-02-19T13:36:46.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117001 2026-02-19T13:36:46.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836496 1-115964117001' 2026-02-19T13:36:46.687 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:36:46.687 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:36:46.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528322 2026-02-19T13:36:46.759 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528322 2026-02-19T13:36:46.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836496 1-115964117001 2-193273528322' 2026-02-19T13:36:46.760 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:46.760 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836496 2026-02-19T13:36:46.760 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:46.761 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:36:46.762 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836496 2026-02-19T13:36:46.762 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:46.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836496 2026-02-19T13:36:46.763 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836496' 2026-02-19T13:36:46.763 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.0 seq 21474836496 2026-02-19T13:36:46.764 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:36:46.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836495 -lt 21474836496 2026-02-19T13:36:46.920 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:36:47.921 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:36:47.922 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:36:48.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836496 -lt 21474836496 2026-02-19T13:36:48.084 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:48.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-115964117001 2026-02-19T13:36:48.084 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:48.086 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:36:48.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-115964117001 2026-02-19T13:36:48.086 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:48.087 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964117001 2026-02-19T13:36:48.088 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 115964117001' 2026-02-19T13:36:48.088 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.1 seq 115964117001 2026-02-19T13:36:48.088 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:36:48.260 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964117001 -lt 115964117001 2026-02-19T13:36:48.261 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:36:48.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-193273528322 2026-02-19T13:36:48.261 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:36:48.262 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:36:48.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-193273528322 2026-02-19T13:36:48.263 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:36:48.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=193273528322 2026-02-19T13:36:48.264 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 193273528322' 2026-02-19T13:36:48.264 INFO:tasks.workunit.client.0.trial144.stderr:waiting osd.2 seq 193273528322 2026-02-19T13:36:48.264 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:36:48.429 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 193273528322 -lt 193273528322 2026-02-19T13:36:48.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:36:48.430 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:48.431 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:48.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-19T13:36:48.620 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:36:48.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:36:48.620 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:36:48.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:36:48.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:36:48.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:36:48.621 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:36:48.774 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T13:36:48.774 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:36:48.775 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:48.775 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:48.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T13:36:48.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-19T13:36:48.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-19T13:36:48.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-19T13:36:48.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T13:36:48.961 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-19T13:36:49.062 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T13:36:49.063 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:36:49.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:36:49.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:36:49.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:36:49.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:36:49.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:36:49.063 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:36:49.217 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T13:36:49.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:36:49.217 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:49.218 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:49.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T13:36:49.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-19T13:36:49.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T13:36:49.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T13:36:49.404 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T13:36:49.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T13:36:49.405 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T13:36:49.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T13:36:49.405 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T13:36:49.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T13:36:49.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T13:36:49.597 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-19T13:36:49.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T13:36:49.598 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-19T13:36:49.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T13:36:49.797 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:36:49.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:36:49.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:36:49.798 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:36:49.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:36:49.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:36:49.799 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:36:49.953 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T13:36:49.953 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:36:49.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:49.954 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:50.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T13:36:50.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-19T13:36:50.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T13:36:50.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T13:36:50.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T13:36:50.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T13:36:50.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T13:36:50.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T13:36:50.144 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T13:36:50.331 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T13:36:50.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T13:36:50.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-19T13:36:50.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T13:36:50.332 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-19T13:36:50.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T13:36:50.733 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:36:50.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:36:50.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:36:50.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:36:50.733 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:36:50.734 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:36:50.734 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:36:50.889 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-19T13:36:50.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:36:50.889 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:50.890 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:51.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-19T13:36:51.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-19T13:36:51.080 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-19T13:36:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-19T13:36:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-19T13:36:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-19T13:36:51.081 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-19T13:36:51.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-19T13:36:51.081 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-19T13:36:51.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-19T13:36:51.274 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-19T13:36:51.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 13 )) 2026-02-19T13:36:51.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-19T13:36:51.275 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-19T13:36:52.075 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-19T13:36:52.076 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:36:52.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:36:52.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:36:52.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:36:52.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:36:52.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:36:52.077 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:36:52.236 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-19T13:36:52.237 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:36:52.237 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:36:52.237 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:36:52.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-19T13:36:52.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:36:52.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:36:52.426 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:70: TEST_recover_unexpected: sleep 5 2026-02-19T13:36:57.427 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:72: TEST_recover_unexpected: ceph pg repair 1.0 2026-02-19T13:36:57.575 INFO:tasks.workunit.client.0.trial144.stderr:instructing pg 1.0 on osd.1 to repair 2026-02-19T13:36:57.586 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:74: TEST_recover_unexpected: sleep 10 2026-02-19T13:37:07.588 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:76: TEST_recover_unexpected: ceph log last 2026-02-19T13:37:07.772 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:36:58.471002+0000 osd.1 (osd.1) 3 : cluster [ERR] 1.0 shard 1 1:602f83fe:::foo:1 : missing 2026-02-19T13:37:07.772 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:36:58.471089+0000 osd.1 (osd.1) 4 : cluster [ERR] repair 1.0 1:602f83fe:::foo:1 : is an unexpected clone 2026-02-19T13:37:07.773 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:36:58.471199+0000 osd.1 (osd.1) 5 : cluster [ERR] 1.0 repair 1 missing, 0 inconsistent objects 2026-02-19T13:37:07.773 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:36:58.471212+0000 osd.1 (osd.1) 6 : cluster [ERR] 1.0 repair 1 missing, 0 inconsistent objects 2026-02-19T13:37:07.773 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:36:58.471237+0000 osd.1 (osd.1) 7 : cluster [ERR] 1.0 repair 3 errors, 2 fixed 2026-02-19T13:37:07.773 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:36:59.491242+0000 osd.1 (osd.1) 9 : cluster [ERR] 1.0 shard 1 soid 1:602f83fe:::foo:1 : candidate size 0 info size 15853 mismatch 2026-02-19T13:37:07.773 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:36:59.491311+0000 osd.1 (osd.1) 10 : cluster [ERR] 1.0 shard 0 soid 1:602f83fe:::foo:1 : data_digest 0x8af68404 != data_digest 0x36b9d416 from auth oi 1:602f83fe:::foo:1(17'2 client.4156.0:1 dirty|data_digest s 15853 uv 1 dd 36b9d416 alloc_hint [0 0 0]) 2026-02-19T13:37:07.773 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:36:59.491313+0000 osd.1 (osd.1) 11 : cluster [ERR] 1.0 shard 1 soid 1:602f83fe:::foo:1 : data_digest 0xffffffff != data_digest 0x8af68404 from shard 0, data_digest 0xffffffff != data_digest 0x36b9d416 from auth oi 1:602f83fe:::foo:1(17'2 client.4156.0:1 dirty|data_digest s 15853 uv 1 dd 36b9d416 alloc_hint [0 0 0]), size 0 != size 15853 from auth oi 1:602f83fe:::foo:1(17'2 client.4156.0:1 dirty|data_digest s 15853 uv 1 dd 36b9d416 alloc_hint [0 0 0]), size 0 != size 15853 from shard 0 2026-02-19T13:37:07.773 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:36:59.491314+0000 osd.1 (osd.1) 12 : cluster [ERR] 1.0 soid 1:602f83fe:::foo:1 : data_digest 0x36b9d416 != data_digest 0x8af68404 from shard 0 2026-02-19T13:37:07.773 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:36:59.491393+0000 osd.1 (osd.1) 13 : cluster [ERR] deep-scrub 1.0 1:602f83fe:::foo:1 : is an unexpected clone 2026-02-19T13:37:07.773 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:36:59.491572+0000 osd.1 (osd.1) 14 : cluster [ERR] 1.0 deep-scrub 0 missing, 1 inconsistent objects 2026-02-19T13:37:07.773 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:36:59.491574+0000 osd.1 (osd.1) 15 : cluster [ERR] 1.0 deep-scrub 4 errors 2026-02-19T13:37:07.773 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:37:00.991483+0000 mon.a (mon.0) 257 : cluster [ERR] Health check failed: 3 scrub errors (OSD_SCRUB_ERRORS) 2026-02-19T13:37:07.774 INFO:tasks.workunit.client.0.trial144.stdout:2026-02-19T13:37:00.991511+0000 mon.a (mon.0) 258 : cluster [ERR] Health check failed: Possible data damage: 1 pg inconsistent (PG_DAMAGED) 2026-02-19T13:37:07.781 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:79: TEST_recover_unexpected: timeout 60 ceph tell osd.0 version 2026-02-19T13:37:07.847 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:37:07.848 INFO:tasks.workunit.client.0.trial144.stdout: "version": "20.2.0-720-g2fc421c8", 2026-02-19T13:37:07.848 INFO:tasks.workunit.client.0.trial144.stdout: "release": "tentacle", 2026-02-19T13:37:07.848 INFO:tasks.workunit.client.0.trial144.stdout: "release_type": "stable" 2026-02-19T13:37:07.848 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:37:07.855 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:80: TEST_recover_unexpected: timeout 60 ceph tell osd.1 version 2026-02-19T13:37:07.919 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:37:07.919 INFO:tasks.workunit.client.0.trial144.stdout: "version": "20.2.0-720-g2fc421c8", 2026-02-19T13:37:07.919 INFO:tasks.workunit.client.0.trial144.stdout: "release": "tentacle", 2026-02-19T13:37:07.919 INFO:tasks.workunit.client.0.trial144.stdout: "release_type": "stable" 2026-02-19T13:37:07.919 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:37:07.926 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:81: TEST_recover_unexpected: timeout 60 ceph tell osd.2 version 2026-02-19T13:37:07.990 INFO:tasks.workunit.client.0.trial144.stdout:{ 2026-02-19T13:37:07.990 INFO:tasks.workunit.client.0.trial144.stdout: "version": "20.2.0-720-g2fc421c8", 2026-02-19T13:37:07.991 INFO:tasks.workunit.client.0.trial144.stdout: "release": "tentacle", 2026-02-19T13:37:07.991 INFO:tasks.workunit.client.0.trial144.stdout: "release_type": "stable" 2026-02-19T13:37:07.991 INFO:tasks.workunit.client.0.trial144.stdout:} 2026-02-19T13:37:07.998 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:35: run: teardown td/osd-unexpected-clone 2026-02-19T13:37:07.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-unexpected-clone 2026-02-19T13:37:07.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:37:07.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-unexpected-clone KILL 2026-02-19T13:37:07.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:37:07.999 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:37:07.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:37:07.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:37:07.999 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:37:08.108 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:37:08.108 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:37:08.109 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:37:08.109 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:37:08.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:37:08.111 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:37:08.111 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:37:08.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:37:08.112 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:37:08.113 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:37:08.113 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:37:08.114 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:37:08.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:37:08.115 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-unexpected-clone 2026-02-19T13:37:08.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:37:08.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:37:08.130 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:37:08.130 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.658971 2026-02-19T13:37:08.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:37:08.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:37:08.132 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-19T13:37:08.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/osd-unexpected-clone 0 2026-02-19T13:37:08.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-unexpected-clone 2026-02-19T13:37:08.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-19T13:37:08.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-unexpected-clone KILL 2026-02-19T13:37:08.133 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:37:08.133 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:37:08.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:37:08.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:37:08.133 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:37:08.134 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:37:08.135 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:37:08.136 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:37:08.136 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:37:08.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:37:08.137 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:37:08.137 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:37:08.138 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:37:08.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:37:08.139 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:37:08.139 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:37:08.140 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:37:08.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-19T13:37:08.141 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-unexpected-clone 2026-02-19T13:37:08.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:37:08.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:37:08.143 INFO:tasks.workunit.client.0.trial144.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.658971 2026-02-19T13:37:08.143 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.658971 2026-02-19T13:37:08.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:37:08.144 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:37:08.145 INFO:tasks.workunit.client.0.trial144.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-19T13:37:08.146 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-19T13:37:08.146 DEBUG:teuthology.orchestra.run.trial144:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-19T13:37:08.194 INFO:tasks.workunit:Stopping ['scrub'] on client.0... 2026-02-19T13:37:08.195 DEBUG:teuthology.orchestra.run.trial144:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-19T13:37:08.585 DEBUG:teuthology.parallel:result is None 2026-02-19T13:37:08.585 DEBUG:teuthology.orchestra.run.trial144:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-19T13:37:08.594 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-19T13:37:08.594 DEBUG:teuthology.orchestra.run.trial144:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-19T13:37:08.638 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-19T13:37:08.638 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-19T13:37:08.653 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-19T13:37:08.654 DEBUG:teuthology.orchestra.run.trial144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-19T13:37:08.786 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-19T13:37:08.797 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-19T13:37:08.797 DEBUG:teuthology.orchestra.run.trial144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-19T13:37:08.859 INFO:teuthology.orchestra.run.trial144.stdout: remote refid st t when poll reach delay offset jitter 2026-02-19T13:37:08.859 INFO:teuthology.orchestra.run.trial144.stdout:============================================================================== 2026-02-19T13:37:08.860 INFO:teuthology.orchestra.run.trial144.stdout:+hou1.us.ntp.li 192.58.120.41 2 u 100 128 377 37.395 -1.539 2.965 2026-02-19T13:37:08.860 INFO:teuthology.orchestra.run.trial144.stdout:*96-19-94-82.cpe .GPS3. 1 u 44 128 377 53.761 -1.587 0.646 2026-02-19T13:37:08.860 INFO:teuthology.orchestra.run.trial144.stdout:-clover0.mattnor 192.126.175.149 3 u 87 128 377 41.388 -3.350 0.465 2026-02-19T13:37:08.860 INFO:teuthology.orchestra.run.trial144.stdout:+ntp1.wiktel.com .PPS. 1 u 27 128 377 38.741 -0.411 0.453 2026-02-19T13:37:08.860 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-19T13:37:08.872 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-19T13:37:08.873 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-19T13:37:08.887 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-19T13:37:08.904 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-19T13:37:08.919 INFO:teuthology.task.internal:Duration was 3381.190781 seconds 2026-02-19T13:37:08.919 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-19T13:37:08.934 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-19T13:37:08.934 DEBUG:teuthology.orchestra.run.trial144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-19T13:37:08.961 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-19T13:37:08.961 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial144.front.sepia.ceph.com 2026-02-19T13:37:08.962 DEBUG:teuthology.orchestra.run.trial144:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-19T13:37:09.008 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-19T13:37:09.008 DEBUG:teuthology.orchestra.run.trial144:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-19T13:37:09.256 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-19T13:37:09.256 DEBUG:teuthology.orchestra.run.trial144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-19T13:37:09.266 INFO:teuthology.orchestra.run.trial144.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-19T13:37:09.267 INFO:teuthology.orchestra.run.trial144.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-19T13:37:09.267 INFO:teuthology.orchestra.run.trial144.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-19T13:37:09.267 INFO:teuthology.orchestra.run.trial144.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-19T13:37:09.267 INFO:teuthology.orchestra.run.trial144.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-19T13:37:09.327 INFO:teuthology.orchestra.run.trial144.stderr: 81.7% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-19T13:37:09.329 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-19T13:37:09.346 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-19T13:37:09.347 DEBUG:teuthology.orchestra.run.trial144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-19T13:37:09.377 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-19T13:37:09.394 DEBUG:teuthology.orchestra.run.trial144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:37:09.426 INFO:teuthology.orchestra.run.trial144.stdout:kernel.core_pattern = core 2026-02-19T13:37:09.434 DEBUG:teuthology.orchestra.run.trial144:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:37:09.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T13:37:09.482 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-19T13:37:09.498 INFO:teuthology.task.internal:Transferring archived files... 2026-02-19T13:37:09.501 DEBUG:teuthology.misc:Transferring archived files from trial144:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/nmordech-2026-02-19_12:10:46-rados-wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle-distro-default-trial/58618/remote/trial144 2026-02-19T13:37:09.502 DEBUG:teuthology.orchestra.run.trial144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-19T13:37:09.560 INFO:teuthology.task.internal:Removing archive directory... 2026-02-19T13:37:09.560 DEBUG:teuthology.orchestra.run.trial144:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-19T13:37:09.590 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-19T13:37:09.607 INFO:teuthology.task.internal:Not uploading archives. 2026-02-19T13:37:09.607 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-19T13:37:09.619 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-19T13:37:09.619 DEBUG:teuthology.orchestra.run.trial144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-19T13:37:09.633 INFO:teuthology.orchestra.run.trial144.stdout: 1179687 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 19 13:37 /home/ubuntu/cephtest 2026-02-19T13:37:09.633 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-19T13:37:09.649 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-19T13:37:09.683 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} duration: 3381.1907811164856 flavor: default owner: scheduled_nmordech@soko04.front.sepia.ceph.com success: true 2026-02-19T13:37:09.683 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-19T13:37:09.806 INFO:teuthology.run:pass